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CONDITIONS 



(57) Abstract 

Methods are provided for developing medical diagnostic tests using deci- 
sion-support systems, such as neural networks. Patient data or information, typi- 
cally patient history or clinical data, are analyzed by the decision-support systems 
to identify important or relevant variables and decision-support systems are trained 
on the patient data. Patient data are augmented by biochemical test data, or results, 
where available, to refine performance. The resulting decision-support systems are 
employed to evaluate specific observation values and test results, to guide the devel- 
opment of biochemical or other diagnostic tests, to assess a course of treatment, to 
identify new diagnostic tests and disease markers, to identify useful therapies, and 
to provide the decision-support functionality for the test. Methods for identifica- 
tion of important input variables for a medical diagnostic test for use in training the 
decision-support systems to guide the development of the tests, for improving the 
sensitivity and specificity of such tests, and for selecting diagnostic tests that improve 
overall diagnosis of, or potential for, a disease state and that permit the effective- 
ness of a selected therapeutic protocol to be assessed are provided. The methods for 
identification can be applied in any field in which statistics are used to determine 
outcomes. A method for evaluating the effectiveness of any given diagnostic test is 
also provided. 



PATIENT HISTORY 
DATA 



ISOLATE IMPORTANT 
VARIABLES 



TRAIN FINAL NETWORKS 




COMPLETED PATIENT 
HISTORY DIAGNOSTIC TEST , 



FOR THE PURPOSES OF INFORMATION ONLY 
Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT\ 



AL 


Albania 


ES 


Spain 


LS 


Lesotho 


SI 


Slovenia 


AM 


Armenia 


FI 


Finland 


LT 


Lithuania 


SK 


Slovakia 


AT 


Austria 


FR 


France 


LU 


Luxembourg 


SN 


Senegal 


AU 


Australia 


GA 


Gabon 


LV 


Latvia 


sz 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


BB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 


GN 


Guinea 


MK 


The former Yugoslav 


TM 


Turkmenistan 


BF 


Burkina Faso 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


Bulgaria 


HU 


Hungary 


ML 


Mali 


TT 


Trinidad and Tobago 


BJ 


Benin 


IE 


Ireland 


MN 


Mongolia 


UA 


Ukraine 


BR 


Brazil 


IL 


Israel 


MR 


Mauritania 


UC 


Uganda 


BY 


Belarus 


IS 


Iceland 


MW 


Malawi 


US 


United States of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


uz 


Uzbekistan 


CF 


Central African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CG 


Congo 


KE 


Kenya 


NL 


Netherlands 


YU 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


Norway 


ZVY 


Zimbabwe 


CI 


Cote d't voire 


KP 


Democratic People's 


NZ 


New Zealand 






CM 


Cameroon 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


PT 


Portugal 






cu 


Cuba 


KZ 


KazaJcstan 


RO 


Romania 






cz 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






DE 


Germany 


LI 


Liechtenstein 


SD 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Eaton i a 


LR 


Liberia 


SG 


Singapore 








WO 99/09507 PCT/US98/1 689 1 

-1- 

METHODS FOR SELECTING, DEVELOPING AND IMPROVING DIAGNOSTIC 
TESTS FOR PREGN ANY-RELATED CONDITIONS 

For international purposes, benefit of priority to U.S. application Serial 

No. 08/912,133, entititled "METHOD FOR SELECTING MEDICAL AND 

5 BIOCHEMICAL DIAGNOSTIC TESTS USING NEURAL NETWORK-RELATED 

APPLICATIONS" to Jerome Lapointe and Duane DeSieno, filed August 14, 1997 

is claimed herein. 

This application is also related to U.S. application Serial 

08/798,306 entitled "METHOD FOR SELECTING MEDICAL AND BIOCHEMICAL 

10 DIAGNOSTIC TESTS USING NEURAL NETWORK-RELATED APPLICATIONS" to 
Jerome Lapointe and Duane DeSieno, filed February 7, 1997. This application 
is also a related to U.S. application Serial 08/599,275 and to International PCT 
application No.PCT/US97/02104 (published as WO 97/29447 published on 14 
August 1997), each entitled "METHOD FOR DEVELOPING MEDICAL AND 

15 BIOCHEMICAL DIAGNOSTIC TESTS USING NEURAL NETWORKS" to Jerome 
Lapointe and Duane DeSieno, filed February 9, 1997. U.S. application Serial 
No. 08/798,306 is a continuation-in-part of U.S. application Serial No. 
08/599,275. U.S. application Serial 

08/599,275, entitled "METHOD FOR DEVELOPING MEDICAL AND 
20 BIOCHEMICAL DIAGNOSTIC TESTS USING NEURAL NETWORKS" to Jerome 
Lapointe and Duane DeSieno, filed February 9, 1996 claims priority under 35 
U.S.C. §1 19(e) to U.S. provisional application Serial No. 60/01 1,449, entitled 
"METHOD AND APPARATUS FOR AIDING IN THE DIAGNOSIS OF 
ENDOMETRIOSIS USING A PLURALITY OF PARAMETERS SUITED FOR 
25 ANALYSIS THROUGH A NEURAL NETWORK" to Jerome Lapointe and Duane 
DeSieno, filed February 9, 1996. 

The subject matter of each of the above-noted applications and 
provisional application is herein incorporated in its entirety by reference thereto. 
FIELD OF THE INVENTION 
30 This subject matter of the invention relates to the use of prediction 

technology, particularly nonlinear prediction technology, for the development of 
medical diagnostic aids for pregnancy-related and fertility-related conditions. In 
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particular, training techniques operative on neural networks and other expert 
systems with inputs from patient historical information for the development of 
medical diagnostic tools and methods of diagnosis are provided. 
BACKGROUND OF THE INVENTION 
5 Data Mining, decision support-systems and neural networks 

A number of computer decision-support systems have the ability to 
classify information and identify patterns in input data, and are particularly 
useful in evaluating data sets having large quantities of variables and complex 
interactions between variables. These computer decision systems which are 

10 collectively identified as "data mining" or "knowledge discovery in databases" 
(and herein as decision-support systems) rely on similar basic hardware 
components, e.g. , personal computers (PCS) with a processor, internal and 
peripheral devices, memory devices and input/output interfaces. The 
distinctions between the systems arise within the software, and more 

15 fundamentally, the paradigms upon which the software is based. Paradigms 
that provide decision-support functions include regression methods, decision 
trees, discriminant analysis, pattern recognition, Bayesian decision theory, and 
fuzzy logic. One of the more widely used decision-support computer systems is 
the artificial neural network. 

20 Artificial neural networks or "neural nets" are parallel information 

processing tools in which individual processing elements called neurons are 
arrayed in layers and furnished with a large number of interconnections between 
elements in successive layers. The functioning of the processing elements are 
modeled to approximate biologic neurons where the output of the processing 

25 element is determined by a typically non-linear transfer function. In a typical 
model for neural networks, the processing elements are arranged into an input 
layer for elements which receive inputs, an output layer containing one or more 
elements which generate an output, and one or more hidden layers of elements 
therebetween. The hidden layers provide the means by which non-linear 

30 problems may be solved. Within a processing element, the input signals to the 
element are weighted arithmetically according to a weight coefficient associated 
with each input. The resulting weighted sum is transformed by a selected non- 
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linear transfer function, such as a sigmoid function, to produce an output, 
whose values range from 0 to 1 , for each processing element. The learning 
process, called "training", is a trial-and-error process involving a series of 
iterative adjustments to the processing element weights so that a particular 
5 processing element provides an output which, when combined with the outputs 
of other processing elements, generates a result which minimizes the resulting 
error between the outputs of the neural network and the desired outputs as 
represented in the training data. Adjustment of the element weights are 
triggered by error signals. Training data are described as a number of training 

10 examples in which each example contains a set of input values to be presented 
to the neural network and an associated set of desired output values. 

A common training method is backpropagation or "backprop", in which 
error signals are propagated backwards through the network. The error signal is 
used to determine how much any given element's weight is to be changed and 

15 the error gradient, with the goal being to converge to a global minimum of the 
mean squared error. The path toward convergence, i.e. , the gradient descent, 
is taken in steps, each step being an adjustment of the input weights of the 
processing element. The size of each step is determined by the learning rate. 
The slope of the gradient descent includes flat and steep regions with valleys 

20 that act as local minima, giving the false impression that convergence has been 
achieved, leading to an inaccurate result. 

Some variants of backprop incorporate a momentum term in which a 
proportion of the previous weight-change value is added to the current value. 
This adds momentum to the algorithm's trajectory in its gradient descent, which 

25 may prevent it from becoming "trapped" in local minima. One backpropogation 
method which includes a momentum term is "Quickprop", in which the 
momentum rates are adaptive. The Quickprop variation is described by Fahlman 
(see,"Fast Learning Variations on Back-Propagation: An Empirical Study", 
Proceedings on the 1988 Connectionist Models Summer School , Pittsburgh, 

30 1988, D. Touretzky, et aL, eds., pp. 38-51, Morgan Kaufmann, San Mateo, CA; 
and, with Lebriere, "The Cascade-Correlation Learning Architecture", Advances 
in Neural Information Processing Systems 2 . (Denver. 1989), D. Touretzky, ed., 
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pp. 524-32. Morgan Kaufmann, San Mateo, CA). The Quickprop algorithm is 
publicly accessible, and may be downloaded via the Internet, from the Artificial 
Intelligence Repository maintained by the School of Computer Science at 
Carnegie Mellon University. In Quickprop, a dynamic momentum rate is 
5 calculated based upon the slope of the gradient. If the slope is smaller but has 
the same sign as the slope following the immediately preceding weight 
adjustment, the weight change will accelerate. The acceleration rate is 
determined by the magnitude of successive differences between slope values. 
If the current slope is in the opposite direction from the previous slope, the 
10 weight change decelerates. The Quickprop method improves convergence 
speed, giving the steepest possible gradient descent, helping to prevent 
convergence to a local minimum. 

When neural networks are trained on sufficient training data, the neural 
network acts as an associative memory that is able to generalize to a correct 
15 solution for sets of new input data that were not part of the training data. 

Neural networks have been shown to be able to operate even in the absence of 
complete data or in the presence of noise. It has also been observed that the 
performance of the network on new or test data tends to be lower than the 
performance on training data. The difference in the performance on test data 
20 indicates the extent to which the network was able to generalize from the 

training data. A neural network, however, can be retrained and thus learn from 
the new data, improving the overall performance of the network. 

Neural nets, thus, have characteristics that make them well suited for a 
large number of different problems, including areas involving prediction, such as 
25 medical diagnosis. 

Neural Nets and Diagnosis 

In diagnosing and/or treating a patient, a physician will use patient 
condition, symptoms, and the results of applicable medical diagnostic tests to 
identify the disease state or condition of the patient. The physician must 
30 carefully determine the relevance of the symptoms and test results to the 
particular diagnosis and use judgement based on experience and intuition in 
making a particular diagnosis. Medical diagnosis involves integration of 
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information from several sources including a medical history, a physical exam 
and biochemical tests. Based upon the results of the exam and tests and 
answers to the questions, the physician, using his or her training, experience 
and knowledge and expertise, formulates a diagnosis. A final diagnosis may 
5 require subsequent surgical procedures to verify or to formulate. Thus, the 
process of diagnosis involves a combination of decision-support, intuition and 
experience. The validity of a physician's diagnosis is very dependent upon 
his/her experience and ability. 

Because of the predictive and intuitive nature of medical diagnosis, 

10 attempts have been made to develop neural networks and other expert systems 
that aid in this process. The application of neural networks to medical diagnosis 
has been reported. For example, neural networks have been used to aid in the 
diagnosis of cardiovascular disorders (see, e.g. . Baxt (1991) "Use of an Artificial 
Neural Network for the Diagnosis of Myocardial Infarction," Annals of Internal 

15 Medicine 1 1 5 :843; Baxt (1992) "Improving the Accuracy of an Artificial Neural 
Network Using Multiple Differently Trained Networks," Neural Computation 
4:772; Baxt (1992), "Analysis of the clinical variables that drive decision in an 
artificial neural network trained to identify the presence of myocardial 
infarction," Annals of Emergency Medicine 21 :1439; and Baxt (19941 

20 "Complexity, chaos and human physiology: the justification for non-linear neural 
computational analysis/' Cancer Letters 77 :85). Other medical diagnostic 
applications include the use of neural networks for cancer diagnosis (see, e.g. , 
Maclin, et aL (19910 "Using Neural Networks to Diagnose Cancer" Journal of 
Medical Systems 1 5:1 1-9; Rogers, et aL (1994) "Artificial Neural Networks for 

25 Early Detection and Diagnosis of Cancer" Cancer Letters 77 :79-83; Wilding, et 
al. (1994) "Application of Backpropogation Neural Networks to Diagnosis of 
Breast and Ovarian Cancer" Cancer Letters 77 :145-53), neuromuscular 
disorders (Pattichis, et aL (1995) "Neural Network Models in EMG Diagnosis", 
IEEE Transactions on Biomedical Engineering 42:5:486-495), and chronic 

30 fatigue syndrome (Solms, et aL (1996) "A Neural Network Diagnostic Tool for 
the Chronic Fatigue Syndrome", International Conference on Neural Networks, 
Paper No. 108). These methodologies, however, fail to address significant 
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issues relating to the development of practical diagnostic tests for a wide range 
of conditions and does not address the selection of input variables. 

Computerized decision-support methods other than neural networks have 
been reported for their applications in medical diagnostics, including knowledge- 
5 based expert systems, including MYCIN (Davis, et aL , "Production Systems as a 
Representation for a Knowledge-based Consultation Program", Artificial 
Intelligence , 1977; 8: 1: 15-45) and its progeny TEIRESIAS, EMYCIN, PUFF, 
CENTAUR, VM, GUIDON, SACON, ONCOCIN and ROGET. MYCIN is an 
interactive program that diagnoses certain infectious diseases and prescribes 

10 anti-microbial therapy. Such knowledge-based systems contain factual 

knowledge and rules or other methods for using that knowledge, with all of the 
information and rules being pre-programmed into the system's memory rather 
than the system developing its own procedure for reaching the desired result 
based upon input data, as in neural networks. Another computerized diagnosis 

15 method is the Bayesian network, also known as a belief or causal probabilistic 
network, which classifies patterns based on probability density functions from 
training patterns and a priori information. Bayesian decision systems are 
reported for uses in interpretation of mammograms for diagnosing breast cancer 
(Roberts, et aL , "MammoNet: A Bayesian Network diagnosing Breast Cancer", 

20 Midwest Artificial Intelligence and Cognitive Science Society Conference, 
Carbondale, IL, April 1995) and hypertension (Blinowska, et aL (1993) 
"Diagnostica A Bayesian Decision-Aid System - Applied to Hypertension 
Diagnosis", IEEE Transactions on Biomedical Engineering 40:230-35) Bayesian 
decision systems are somewhat limited in their reliance on linear relationships 

25 and in the number of input data points that can be handled, and may not be as • 
well suited for decision-support involving non-linear relationships between 
variables. Implementation of Bayesian methods using the processing elements 
of a neural network can overcome some of these limitations (see, e.g. . Penny, 
et aL (1996) In "Neural Networks in Clinical Medicine", Medical Decision- 

30 support , 1996; 16:4: 386-98). These methods have been used, by mimicking 
the physician, to diagnose disorders in which important variables are input into 
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the system. It, however, would be of interest to use these systems to improve 

upon existing diagnostic procedures. 

Preterm delivery and other pregnancy-related conditions 

Determination of impending preterm births and the risk of preterm births 
5 is critical for increasing neonatal survival of preterm infants. Many methods for 
detecting or predicting the risk of preterm birth and/or the risk of impending 
preterm delivery are subjective, not sufficiently sensitive, and not specific. In 
particular, preterm neonates account for more than half, and maybe as many as 
three-quarters of the 

10 morbidity and mortality of newborns without congenital anomalies. Although 
tocolytic agents that delay delivery were introduced 20 to 30 years ago, there 
has been only a minor decrease in the incidence of preterm delivery. It has been 
postulated that the failure to observe a larger reduction in the incidence of 
preterm births is due to errors in the 

15 diagnosis of preterm labor and the risk of preterm delivery and because the 

conditions are too advanced by the time they are recognized for tocolytic agents 
to successfully delay the birth. 

There are a number of biochemical tests for assessing the risk of preterm 
delivery and other traditional methods of diagnosis based on symptomologies. 

20 These methods have false-negative and false-positive error rates. Traditional 
diagnosis also can require subjective interpretation and may require 
sophisticated training or equipment. The validity of the diagnosis is related to 
the experience and ability of the physician. Thus, there is a need for improved 
methods for assessing risk of preterm delivery, predicting imminent delivery and 

25 assessing time of delivery. 

Therefore, it is an object herein to provide a non-invasive 
diagnostic aid for assessing the risk of preterm delivery. It is also an object 
herein to identify new variables, identify new biochemical tests and markers for 
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preterm delivery and to design to new diagnostic tests that improve upon 
existing diagnostic methodologies. 
SUMMARY OF THE INVENTION 

Methods using decision-support systems for the diagnosis of and for 
5 aiding in the diagnosis of diseases, disorders and other medical conditions are 
provided. In particular, methods provided herein, assess the risk of preterm 
delivery and also the risk of delivery in a selected period of time (delivery-related 
risks). These methods are useful for assessing these risks in symptomatic 
pregnant female mammals, particularly human females. 

10 Also provided are methods that use patient history data and identification 

of important variables to develop a diagnostic test for these assessing these 
delivery-related risks; a method for identification of important selected variables 
for use in assessing these delivery-related risks; a method of designing a 
diagnostic test for assessing; a method of evaluating the usefulness of 

15 diagnostic test for these assessments; a method of expanding clinical utility of a 
diagnostic test to include assessment of these delivery-related risks, and a 
method of selecting a course of treatment to reduce the risk of delivery within a 
selected period of time or preterm by predicting the outcome of various possible 
treatments. 

20 Also provided are disease parameters or variables to aid in predicting 

pregnancy-related events, such as the likelihood of delivery within a particular 
time period, and for assessing the risk of preterm delivery. 

Also provided are means to use neural network training to guide the 
development of the tests to improve their sensitivity and specificity, and to 

25 select diagnostic tests that improve overall diagnosis of, or potential for, 

assessment of the risk of preterm delivery or delivery within a selected period of 
time. A method for evaluating the effectiveness of any given diagnostic test is 
assessment of the risk of preterm delivery or delivery within a selected period of 
time is also provided. Also provided herein is a method for identifying variables 

30 or sets of variables that aid in the assessment of the risk of preterm delivery or 
delivery within a selected period of time. 
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Methods are provided for developing medical diagnostic tests for 
assessment of the risk of preterm delivery or delivery within a selected period of 
time using computer-based decision-support systems, such as neural networks 
and other adaptive processing systems (collectively, "data mining tools"). The 
5 neural networks or other such systems are trained on the patient data and 
observations collected from a group of test patients in whom the condition is 
known or suspected; a subset or subsets of relevant variables are identified 
through the use of a decision-support system or systems, such as a neural 
network or a consensus of neural networks; and another set of decision-support 

10 systems is trained on the identified subset(s) to produce a consensus decision- 
support system based test, such as a neural net-based test for the condition. 
The use of consensus systems, such as consensus neural networks, minimizes 
the negative effects of local minima in decision-support systems, such as neural 
network-based systems, thereby improving the accuracy of the system. 

15 To refine or improve performance, the patient data can be augmented by 

increasing the number of patients used. Also biochemical test data and other 
data may be included as part of additional examples or by using the data as 
additional variables prior to the variable selection process. 

The resulting systems are used as an aid in assessment of the risk of 

20 preterm delivery or delivery within a selected period of time. In addition, as the 
systems are used patient data can be stored and then used to further train the 
systems and to develop systems that are adapted for a particular genetic 
population. This inputting of additional data into the system may be 
implemented automatically or done manually. By doing so the systems 

25 continually learn and adapt to the particular environment in which they are 

used. The resulting systems have numerous uses in addition to assessment of 
the risk of preterm delivery or delivery within a selected period of time, which 
include predicting the outcome of a selected treatment protocol. The systems 
may also be used to assess the value of other data in a diagnostic procedure, 

30 such as biochemical test data and other such data, and to identify new tests 

that are useful for assessment of the risk of preterm delivery or delivery within a 
selected period of time. 
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The methods are exemplified with reference to neural networks, 
however, it is understood that other data mining tools, such as expert systems, 
fuzzy logic, decision trees, and other statistical decision-support systems which 
are generally non-linear, may be used. Although the variables provided herein 
5 are intended to be used with decision-support systems, once the variables are 
identified, then a person, typically a physician, armed with knowledge the 
important variables can use them to aid in diagnosis in the absence of a 
decision-support system or using a less complex linear system of analysis. 

In the methods for identifying and selection of important variables and 

10 generating systems for diagnosis, patient data or information, typically patient 
history or clinical data that are the answers to particular queries are collected 
and variables based on this data are identified. For example, the data includes 
the answer to a query regarding the number of pregnancies each patient has 
had. The extracted variable is, thus, number of pregnancies and the query is 

15 the how many prior pregnancies (set forth herein as prior pregnancies). The 
variables are analyzed by the decision-support systems, exemplified by neural 
networks, to identify important or relevant variables. 

A plurality of factors, twelve to about sixteen, particularly a set of 
fourteen factors, in a specific trained neural network extracted from a collection 

20 have been identified as indicia for preterm delivery. 

In other embodiments, for example, a method for assessing the risk of 
delivery prior to completion of 35 weeks of gestation, comprising assessing a 
subset of variables containing at least three and up to all of the responses to 
the following queries: Ethnic Origin Caucasian; Marital Status living with 

25 partner; EGA by sonogram; EGA at sampling; estimated date of delivery by 
best; cervical dilatation (CM); parity-preterm; vaginal bleeding at time of 
sampling; cervical consistency at time of sampling; and previous pregnancy 
without complication is provided. The method uses a decision-support system 
that has been trained to assesses the risk of delivery prior to 35 weeks of 

30 gestation. 

A method for assessing the risk for delivery in 7 or fewer days, 
comprising assessing a subset of variables containing at least three up to all of 
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the following variables: Ethnic Origin Caucasian; Uterine contractions with or 
without pain; Parity-abortions; vaginal bleeding at time of sampling; uterine 
contractions per hour; and No previous pregnancies is provided. The method 
uses a decision-support system that has been trained to assesses the risk of 
5 delivery within seven days. 

A method for assessing the risk for delivery in 14 or fewer days, 
comprising assessing a subset of variables containing at least three up to all of 
the following variables: Ethnic Origin Hispanic; Marital Status living with 
partner; Uterine contractions with or without pain; Cervical dilatation; Uterine 

10 contractions per hour; and No previous pregnancies is provided. This method 
uses a decision-support system that has been trained to assess the risk of 
delivery within fourteen days. 

As shown herein, variables or combinations thereof that heretofore were 
not known to be important in aiding in assessment of the risk of preterm 

15 delivery or delivery within a selected period of time are identified. In addition, 
patient history data, without supplementing biochemical test data, can be used 
to diagnose or aid in diagnosing a disorder or condition when used with the 
decision-support systems, such as the neural nets provided herein. 

Also provided herein is a method of identifying and expanding clinical 

20 utility of diagnostic test. The results of a particular test, particular one that had 
heretofore not been considered of clinical utility with respect to assessment of 
the risk of preterm delivery or delivery within a selected period of time, are 
combined with the variables and used with the decision-support system, such 
as a neural net. If the performance, the ability to correctly diagnose a disorder, 

25 of the system is improved by addition of the results of the test, then the test 
will have clinical utility or a new utility is assessing the risk of preterm delivery. 

Similarly, the resulting systems can be used to identify new utilities for 
drugs or therapies and also to identify uses for particular drugs and therapies for 
30 reducing the risk of preterm delivery. For example, the systems can be used to 
select subpopulations of patients for whom a particular drug or therapy is 
effective. Thus, methods for expanding the indication for a drug or therapy 
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and identifying new drugs and therapies are provided. Diagnostic software and 
exemplary neural networks that use the variables for assessment of the risk of 
delivery before a specified time are also provided. 

In other embodiments, the performance of a diagnostic neural network 
5 system for assessing risk of preterm delivery is enhanced by including variables 
based on biochemical test results from a relevant biochemical test as part of the 
factors (herein termed biochemical test data) used for training the network. 
One of exemplary networks described herein that results therefrom is an 
augmented neural network that employs 6 input factors, including results of a 
10 biochemical test and the 7 clinical parameters. The set of weights of the 

augmented neural networks differ from the set of weights of the clinical data 
neural networks. The exemplified biochemical test employs an immuno* 
diagnostic test format, such as the ELISA diagnostic test format. Neural 
networks, thus, can be trained to predict the disease state based on the 
15 identification of factors important in predicting the disease state and combining 
them with biochemical data. 

The resulting diagnostic systems may be adapted and used not only for 
diagnosing the presence of a condition or disorder, but also the severity of the 
disorder and as an aid in selecting a course of treatment. 
20 BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 is a flow chart for developing a patient-history-based 
diagnostic test process. 

FIGURE 2 is a flow chart for developing a biochemical diagnostic test. 

FIGURE 3 is a flow chart of the process for isolating important variables. 
25 FIGURE 4 is a flow chart on the process of training one or a set of neural 

networks involving a partitioning of variables. 

FIGURE 5 is a flow chart for developing a biochemical diagnostic test. 

FIGURE 6 is a flow chart for determining the effectiveness of a 
biochemical diagnostic test. 
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FIGURE 7 depicts an exemplary screen showing main menu, tool bar and 
results display in the user interface using the software for assessing preterm 
delivery; 

FIGURE 8 depicts an exemplary Edit Record dialog box in preterm 
5 delivery software; 

FIGURE 9 depicts an exemplary Go To dialog box in the software; 
FIGURE 10 depicts an exemplary Help About dialog box in the software; 
FIGURES 1 1 A and 1 1B shows exemplary outputs from the software, 
FIGURE 1 1 B includes the input data as well; 
10 FIGURE 12 is a schematic diagram of a neural network (EGA6) trained on 

clinical data of the form used for the consensus network of a plurality of neural 
networks; and 

FIGURE 13 is a schematic diagram of a neural network, such as EGAD7f 
and EGAD14f, trained on clinical data of the form used for the consensus 

15 network of a plurality of neural networks. 

FIGURE 14 is a schematic diagram of a consensus network of eight 
neural networks. A final indicator pair C, D is based on an analysis of a 
consensus of preliminary indicator pairs from a plurality, specifically eight, 
trained neural networks 10A - 10H. Each preliminary indicator pair A, B is 

20 provided to one of two consensus processors 150, 152. via paths 133-140 and 
141-148. The first consensus processor 150 processes all positive indicators. 
The second consensus processor 152 processes all negative indicators. Each 
consensus processor 150, 152 is an averager, i.e., it merely forms a linear 
combination, such as an average, of the collection of like preliminary indicator 

25 pairs A, B. The resultant confidence indicator pair is the desired result, where 
the inputs are the set of clinical factors for the patient under test. 
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
Definitions 

Unless defined otherwise, all technical and scientific terms used herein 
30 have the same meaning as is commonly understood by one of skill in the art to 
which this invention belongs. All patents, applications and publications referred 
to herein are incorporated by reference. 
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As used herein, a decision-support system, also referred to as a "data 
mining system" or a "knowledge discovery in data system", is any system, 
typically a computer-based system, that can be trained on data to classify the 
input data and then subsequently used with new input data to make decisions 
5 based on the training data. These systems include, but are not limited, expert 
systems, fuzzy logic, non-linear regression analysis, multivariate analysis, 
decision tree classifiers, Bayesian belief networks and, as exemplified herein, 
neural networks. 

As used herein, an adaptive machine learning process refers to any 
10 system whereby data are used to generate a predictive solution. Such 

processes include those effected by expert systems, neural networks, and fuzzy 
logic. 

As used herein, expert system is a computer-based problem solving and 
decision-support system based on knowledge of its task and logical rules or 
15 procedures for using the knowledge. Both the knowledge and the logic are 

entered into the computer from the experience of human specialists in the area 
of expertise. 

As used herein, a neural network, or neural net, is a parallel 
computational model comprised of densely interconnected adaptive processing 

20 elements. In the neural network, the processing elements are configured into an 
input layer, an output layer and at least one hidden layer. Suitable neural 
networks are known to those of skill in this art (see, e.g. , U.S. Patents 
5,251,626; 5,473,537; and 5,331,550, Baxt (1991) "Use of an Artificial Neural 
Network for the Diagnosis of Myocardial Infarction," Annals of Internal Medicine 

25 1 15 :843; Baxt (1992) "Improving the Accuracy of an Artificial Neural Network 
Using Multiple Differently Trained Networks," Neural Computation 4:772; Baxt 
(1992) "Analysis of the clinical variables that drive decision in an artificial neural 
network trained to identify the presence of myocardial infarction," Annals of 
Emergency Medicine 21 :1439: and Baxt (1994) "Complexity, chaos and human 

30 physiology: the justification for non-linear neural computational analysis," 
Cancer Letters 77:85). 
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As used herein, a processing element, which may also be known as a 
perceptron or an artificial neuron, is a computational unit which maps input data 
from a plurality of inputs into a single binary output in accordance with a 
transfer function. Each processing element has an input weight corresponding 
5 to each input which is multiplied with the signal received at that input to 
produce a weighted input value. The processing element sums the weighted 
inputs values of each of the inputs to generate a weighted sum which is then 
compared to the threshold defined by the transfer function. 

As used herein, transfer function, also known as a threshold function or 

10 an activation function, is a mathematical function which creates a curve 

defining two distinct categories. Transfer functions may be linear, but, as used 
in neural networks, are more typically non-linear, including quadratic, 
polynomial, or sigmoid functions. 

As used herein, backpropogation, also known as backprop, is a training 

15 method for neural networks for correcting errors between the target output and 
the actual output. The error signal is fed back through the processing layer of 
the neural network, causing changes in the weights of the processing elements 
to bring the actual output closer to the target output. 

As used herein, Quickprop is a backpropogation method that was 

20 proposed, developed and reported by Fahlman {"Fast Learning Variations on 

Back-Propagation: An Empirical Study", Proceedings on the 1988 Connectionist 
Models Summer School , Pittsburgh, 1988, D. Touretzky, et aL, eds., pp.38-51, 
Morgan Kaufmann, San Mateo, CA; and, with Lebriere, "The Cascade- 
Correlation Learning Architecture", Advances in Neural Information Processing 

25 Systems 2 . (Denver. 1989), D. Touretzky, ed., pp. 524-32. Morgan Kaufmann/ 
San Mateo, CA). 

As used herein, diagnosis refers to a predictive process in which the 
presence, absence, severity or course of treatment of a disease, disorder or 
other medical condition is assessed. For purposes herein, diagnosis will also 
30 include predictive processes for determining the outcome resulting from a 
treatment. 
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As used herein, a patient or subject includes any mammals for whom 
diagnosis is contemplated. Humans are the preferred subjects. 

As used herein, biochemical test data refers to the results of any 
analytical methods, which include, but are not limited to:, immunoassays, 
5 bioassays, chromatography, data from monitors, and imagers; measurements 
and also includes data related to vital signs and body function, such as pulse 
rate, temperature, blood pressure, the results of, for example, EKG, ECG and 
EEG, biorhythm monitors and other such information. The analysis can assess 
for example, analytes, serum markers, antibodies, and other such material 
10 obtained from the patient through a sample. 

As used herein, patient historical data refers to data obtained from a 
patient, such as by questionnaire format, but typically does not include 
biochemical test data as used herein, except to the extent such data is 
historical, a desired solution is one that generates a number or result whereby a 
15 diagnosis of a disorder can be generated. 

As used herein, wherein a training example includes the observation data 
for a single diagnosis, typically the observation data related to one patient. 

As used herein, the parameters identified from patient historical data are 
herein termed observation factors or values or variables. For example, patient 
20 data will include information with respect to individual patient's smoking habits. 
The variable associated with that will be smoking. 

As used herein, partition means to select a portion of the data, such as 
80%, and use it for training a neural net and to use the remaining portion as 
test data. Thus, the network is trained on all but one portion of the data. The 
25 process can then be repeated and a second network trained. The process is 
repeated until all partitions are used as used as test data and training data. 

As used herein, the method of training by partitioning the available data 
into a plurality of subsets is generally referred to as the "holdout method" of 
training. The holdout method is particularly useful when the data available for 
30 network training is limited. 

As used herein, training refers to the process in which input data are 
used to generate a decision-support system. In particularly, with reference to 
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neural nets, training is a trial-and-error process involving a series of iterative 
adjustments to the processing element weights so that a particular processing 
element provides an output which, when combined with the outputs of other 
processing elements, generates a result which minimizes the resulting error 
5 between the outputs of the neural network and the desired outputs as 
represented in the training data. 

As used herein, a variable selection process is a systematic method 
- whereby combinations of variables that yield predictive results are selected from 
any available set. Selection is effected by maximizing predictive performance of 
10 subsets such that addition of additional variables does not improve the result. 
The preferred methods provided herein advantageously permit selection of 
variables without considering ail possible combinations. 

As used herein, a candidate variable is a selected item from collected 
observations from a group of test patients for the diagnostic embodiments or 
15 other records, such as financial records, that can be used with the decision- 
support system. Candidate variables will be obtained by collecting data, such 
as patient data, and categorizing the observations as a set of variables. 

As used herein, important selected variables refer to variables that 
enhance the network performance of the task at hand. Inclusion of all available 
20 variables does not result in the optimal neural network; some variables, when 

included in network training, lower the network performance. Networks that are 
trained only with relevant parameters result in increased network performance. 
These variables are also referred to herein as a subset of relevant variables. 

As used herein, ranking refers to a process in which variables are listed 
25 in an order for selection. Ranking may be arbitrary or, preferably, is ordered. 
Ordering may be effected, for example, by a statistical analysis that ranks the 
variables in order of importance with respect to the task, such as diagnosis, or 
by a decision-support system based analysis. Ranking may also be effected, for 
example, by human experts, by rule based systems, or any combination of any 
30 of these methods. 
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As used herein, a consensus of neural networks refers to the linear 
combination of outputs from a plurality of neural networks where the weight on 
each is outputs is determined arbitrarily or set to an equal value. 

As used herein, a greedy algorithm is a method for optimizing a data set 
5 by determining whether to include or exclude a point from a given data set. 
The set begins with no elements and sequentially selects an element from the 
feasible set of remaining elements by myopic optimization, in which, given any 
partial solution, another value that improves the object the most is selected. 
As used herein, a genetic algorithm is a method that begins with an 

10 initial population of randomly generated neural networks which are run through 
a training cycle and ranked according to their performance in reaching the 
desired target. The poor-performing networks are removed from the population, 
with the fitter networks being retained and selected for the crossover process to 
offspring that retain the desirable characteristics of the parent networks. 

15 As used herein, performance of a system is said to be improved or higher 

when the results more accurately predict or determine a particular outcome. It is 
also to be understood that the performance of a system will typically be better 
as more training examples are used. Thus, the systems herein will improve over 
time as they are used and more patient data is accumulated and then added to 

20 the systems as training data. 

As used herein, sensitivity = TP/(TP + FN); specificity is TN/(TN + FP), 
where TP = true positives; TN=true negatives; FP = false positives; and 
FN = false negative. Clinical sensitivity measures how well a test detects 
patients with the disease; clinical specificity measures how well a test correctly 

25 identifies those patients who do not have the disease. 

As used herein, positive predictive value (PPV) is TP/(TP+ FP); and 
negative predictive value (NPV) is TN/(TN + FN). Positive predictive value is the 
likelihood that a patient with a positive test actually has the disease, and 
negative predictive value is the likelihood that a patient with a negative test 

30 result does not have the disease. 
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As used herein, fuzzy logic is an approach to deal with systems that 
cannot be described precisely. Membership functions (membership in a data 
set) are not binary in fuzzy logic systems; instead membership function may 
take on fractional values. Therefore, an element can be simultaneously 
5 contained in two contradictory sets, albeit with different coefficients of set 
membership. Thus, this type of approach is useful for answering questions in 
which there is no yes or answer. Thus, this type of logic is suitable for 
categorizing responses from patient historical questionnaires, in which the 
answer is often one of degree. 

10 As used herein, term refers to delivery at about 40 weeks. 

Preterm delivery refers to delivery prior to that time and, particularly prior to 
completed fetal development. The critical time period with respect to the risk of 
preterm delivery is typically anytime on or before 37 weeks because fetal lung 
development may not be completed. Typically lung maturation to permit the 

15 infant to breathe on its own is complete between about 34 and 37 weeks 

delivery. Thus, the focus of risk assessment is risk of delivery before 37 weeks 
and more particularly before 35 weeks. The earlier the risk of preterm delivery 
can be assessed, the better the opportunity for the clinician to provide 
appropriate care and intervention, where available and possible. 

20 The methods herein are designed to be used at any time during pregnancy. 
The minimum parameters include, for example, those listed in FIG. 8 or 1 1B. 

As used herein, the assessment of the risk of preterm delivery and also 
the risk of delivery in a selected period of time are referred to as "delivery- 
related risks." 

25 As used herein, risk of delivery within a selected period of time refers 

either to prediction of endpoint, he., whether a woman will deliver on or before 
a particular gestational; or regardless of the present gestational age, the risk of 
delivery within a given time interval, such as within 7 days or less, within 14 
days or less or any selected interval. 

30 1 . General considerations and general methodology 

The general methodology relied upon in developing the decision support 
systems provided herein is described in described in co-owned applications U.S. 
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application Serial No. 08/798,306 and published International PCT application 
No. WO 97/29447. 

It has been determined that a number of techniques can be used to train 
neural networks for analyzing observation values such as patient history and/or 
5 biochemical information. Depending upon the characteristics of the available 
data and the problem to be analyzed, different neural network training 
techniques can be used. For example, where large amounts of training inputs 
are available, methodology may be adopted to eliminate redundant training 
information. 

10 Neural networks may also reveal that certain input factors that were not 

initially considered to be important can influence an outcome, as well as reveal 
that presumably important factors are not outcome determinative. The ability of 
neural networks to reveal the relevant and irrelevant input factors permit their 
use in guiding the design of a diagnostic test. As shown herein, neural 

15 networks, and other such data mining tools, are a valuable advance in 

diagnostics, providing an opportunity to increase the sensitivity and specificity 
of a diagnostic test. As shown herein, care must be taken to avoid the 
potential of poor-accuracy answer due to the phenomenon of local minima. The 
methods herein provide a means to avoid this problem or at least minimize it. 

20 In developing the developing diagnostic procedures, and in particular 

diagnostic tests that are based solely or in part on patient information, a number 
of problems have been solved. For example, there is generally a limited amount 
of data because there is a limited number of patients where training data are 
available. To solve this, as described below, the patient information is 

25 partitioned when training the network. Also, there is generally a large number 
of Input observation factors available for use in connection with the available 
data, so methods for ranking and selecting observations were developed. 

Also, there are generally large number of binary (true/false) input factors 
in the available patient data, but these factors are generally sparse in nature 

30 (values that are positive or negative in only a small percentage of cases of the 
binary input factors in the available patient data). Also there is a high degree of 
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overlap between the positive and negative factors of the condition being 
diagnosed. 

These characteristics and others impact the choice of procedures and 
methods used to develop a diagnostic test. These problems are addressed and 
5 solved herein. 

As shown in U.S. application Serial No. 08/798,306 and published 
International PCT application No. WO 97/29447, computer-based decision- 
support systems such as neural networks reveal that certain input factors, 
which were not initially considered to be important, can influence an outcome. 

10 This ability of a neural network to reveal the relevant input factors permits its 
use in guiding the design of diagnostic tests. Thus a method of designing a 
diagnostic test, and a method of evaluating utility of diagnostic test are also 
provided. In each instance, the data from the test or possible test is added to 
the input of the decision-support system. If the results are improved when the 

15 data are included in the input, then the diagnostic test may have clinical utility. 
In this manner, tests that heretofore were not known to be of value in diagnosis 
of a particular disorder are identified, or new tests can be developed. Neural 
networks can add robustness to diagnostic tests by discounting the effects of 
spurious data points and by identifying other data points that might be 

20 substituted, if any. 

Networks are trained on one set of variables and then clinical data from 
diagnostic or biochemical test data and/or additional patient information are 
added to the input data. Any variable that improves the results compared to 
their absence is (are) selected. As a result, particular tests that heretofore were 

25 of unknown value in diagnosing a particular disorder can be shown to have 
relevance. For example, the presence or absence of particular spots on a 
western blot of serum antibodies can be correlated with a disease state. Based 
on the identity of particular spots ( i.e. , antigens) new diagnostic tests can be 
developed. 

30 An example of the application of the prediction technology to aid in the 

diagnosis of disease and more particularly the use of neural network techniques 
with inputs from various information sources to aid in the prediction of time of 
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delivery and assessment of the risk of preterm delivery is provided. A trained 
set of neural networks operative according to a consensus of networks in a 
computer system is employed to evaluate specific clinical associations, for 
example obtained by survey, some of which may not generally be associated 
5 with a disease condition. Exemplary neural networks are provided and factors 
used to aid in the assessment are provided. The neural network training is 
based on correlations between answers to questions furnished by physicians of 
a significant number of clinical patients whose condition was verified. 
2. Development of patient history diagnostic test 
10 Diagnostic tests 

Methods for diagnosis based solely on patient history data are provided. 
As demonstrated herein, it is possible to provide decision-support system that 
rely only on patient history information but that aid in diagnosis. Consequently, 
the resulting systems can then be used to improve the predictive ability of 
15 biochemical test data, to identify new disease markers, to develop biochemical 
tests, to identify tests that heretofore were not thought to be predictive of a 
particular disorder. 

The methods may also be used to select an appropriate course of 
treatment by predicting the result of selected course of treatment and to predict 
20 status following therapy. The input variables for training would be derived 

from, for example, electronic patient records, that indicate diagnoses and other 
available data, including selected treatments and outcomes. The resulting 
decision-support system would then be used with all available data to, for 
example, categorize women into different classes that will respond to different 
25 treatments and predict the outcome of a particular treatment. This permits 
selection of a treatment or protocol most likely to be successful. 

Similarly, the systems can be used to identify new utilities for drugs or 
therapies and also to identify uses for particular drugs and therapies. For 
example, the systems can be used to select subpopulations of patients for 
30 whom a particular drug or therapy is effective. Thus, methods for expanding 

the indication for a drug or therapy and identifying new drugs and therapies are 
provided. 
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Collection of patient data, generation of variables and overview 
To exemplify the methods herein, Fig. 1 sets forth a flow chart for 
developing a patient-history-based diagnostic test process. The process begins 
with collection of patient history data (Step A). Patient history data or 
5 observation values are obtained from patient questionnaires, clinical results, in 
some instances diagnostic test results, and patient medical records and supplied 
in computer-readable form to a system operating on a computer. In the digital 
computer, the patient history data are categorized into a set of variables of two 
forms: binary {such as true/false) values and quantitative (continuous) values. 

10 A binary-valued variable might include the answer to the question, "Do you 
smoke?" A quantitative-valued variable might be the answer to the question, 
"How many packs per day do you smoke?" Other values, such as 
membership functions, may also be useful as input vehicles. 

The patient history data will also include a target or desired outcome 

15 variable that would be assumed to be indicative of the presence, absence, or 
severity of the medical condition to be diagnosed. This desired outcome 
information is useful for neural network training. The selection of data to be 
included in the training data can be made with the knowledge or assumption of 
the presence, severity, or absence of the medical condition to be diagnosed. 

20 As noted herein, diagnosis may also include assessment of the progress and/or 
effectiveness of a therapeutic treatment. 

The number of variables, which can be defined and thus generated, can 
be unwieldy. Binary variables are typically sparse in that the number of positive 
(or negative) responses is often a small percentage of the overall number of 

25 responses. Thus, in instances in which there is a large number of variables and 
a small number of patient cases available in a typical training data environment, 
steps are taken to isolate from the available variables a subset of variables 
important to the diagnosis (Step B). The specific choice of the subset of 
variables from among the available variables will affect the diagnostic 

30 performance of the neural network. 
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The process outlined herein has been found to produce a subset of 
variables which is comparable or superior in sensitivity and reliability to the 
subset of variables typically chosen by a trained human expert, such as a 
physician. In some instances, the variables are prioritized or placed in order of 
5 rank or relevance. 

Thereafter, the final neural networks to be used in the diagnostic 
procedure are trained (Step C). In preferred embodiments, a consensus (i.e. a 
plurality) of networks are trained. The resulting networks form the decision- 
support functionality for the completed patient history diagnostic test (Step D). 
10 Method for isolation of important variables 

A method for isolation of important variables is provided herein. The 
method permits sets of effective variables to be selected without comparing 
every possible combination of variables. The important variables may be used 
as the inputs for the decision-support systems. 
15 Isolation of important or relevant variables -ranking the variables 

Figure 3 provides a flow chart of the process for isolating the important 
or relevant variables (Step E) within a diagnostic test. Such a process is 
typically conducted using a digital computer system to which potentially 
relevant information has been provided. This procedure ranks the variables in 
20 order of importance using two independent methods, then selects a subset of 
the available variables from the uppermost of the ranking. As noted above, 
other ranking methods can be used by those of skill in the art in place of chi 
square or sensitivity analysis. Also, if 

x is set to N (the total number of candidate variables), then ranking can be 
25 arbitrary. 

The system trains a plurality of neural networks on the available data 
(Step I), as explained hereinafter, then generates a sensitivity analysis over all 
trained networks to determine to what extent each input variable was used in 
the network to perform the diagnosis (Step J). A consensus sensitivity analysis 
30 of each input variable is determined by averaging the individual sensitivity 
analysis results for each of the networks trained. Based upon sensitivity, a 
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ranking order for each of the variables available from the patient history 
information is determined (Step K). 

Ranking the variables 
in preferred embodiments, the variables are ranked using a statistical 
5 analysis, such as a chi square analysis, and/or a decision-support system-based 
analysis, such as a sensitivity analysis. A sensitivity analysis and chi square 
analysis are used, in the exemplary embodiment to rank variables. Other 
statistical methods and/or decision-support system-based , including but not 
limited to regression analysis, discriminant analysis and other methods known to 
10 those of skill in the art, may be used. The ranked variables may be used to 
train the networks, or, preferably, used in the method of variable selection 
provided herein. 

The method employs a sensitivity analysis in which each input is varied 
and the corresponding change in output is measured (see, also, Modai, et aL, 

15 (1993) "Clinical Decisions for Psychiatric Inpatients and Their Evaluation by 
Trained Neural Networks", Methods of Information in Medicine 32:396-99; 
Wilding et aL (1994) "Application of Backpropogation Neural Networks to 
Diagnosis of Breast and Ovarian Cancer", Cancer Letters 77 :145-53; Ruck et al. 
(1990) "Feature Selection in Feed-Forward Neural Networks", Neural Network 

20 Computing 20:40-48; and Utans, et aL (1993) "Selecting Neural Network 
Architectures Via the Prediction Risk: Application to Corporate Bond Rating 
Prediction"; Proceedings of the First International Conference on Artificial 
Intelligence Applications on Wall Street. Washington, D.C. , IEEE Computer 
Society Press, pp. 35-41; Penny et aL (1996) In "Neural Networks in Clinical 

25 Medicine", Medical Decision-support 4:386-398). Such methods, which have ■ 
heretofore not been used to select important variables, as described herein. For 
example, sensitivity analysis has bee reported to be used to develop a statistical 
approach to determine the relationships between the variables, but not for 
selection of important variables (see, Baxt et aL (1995) "Bootstrapping 

30 Confidence Intervals for Clinical Input Variable Effects in a Network Trained to 
Identify the Presence of Myocardial Infarction," Neural Computation 7: 624-38). 
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Any such sensitivity analyses may be used as described herein as part of the 
selection of important variables as an aid to diagnosis. 

In a particular embodiment, the sensitivity analysis involves: 
(k) determining an average observation value for each of the variables in an 
5 observation data set; (I) selecting a training example, and running the example 
through a decision-support system to produce an output value, designated and 
stored as the normal output; (m) selecting a first variable in the selected training 
example, replacing the observation value with the average observation value of 
the first variable; running the modified example in the decision-support system 

10 in the forward mode and recording the output as the modified output; In) 

squaring the difference between the normal output and the modified output and 
accumulating it as a total for each variable, in which this total is designed the 
selected variable total for each variable; (o) repeat steps (m) and <n) for each 
variable in the example; (p) repeating steps (l)-(n) for each example in the data 

15 set, where each total for the selected variable represents the relative 

contribution of each variable to the determination of the decision-support 
system output. This total will be used to rank each variable according to its 
relative contribution to the determination of the decision-support system output. 
Step K, Fig. 3, provides an outline of the sensitivity analysis. Each 

20 network or a plurality of trained neural networks (networks N, through N n ) is run 
in the forward mode (no training) for each training example S x (input data group 
for which true output is known or suspected; there must be at least two training 
examples), where "x" is the number of training examples. The output of each 
network for each training example S x is recorded, i.e., stored in memory. 

25 A new training example is defined containing the average value for each input 
variable within all training examples. One at a time, each input variable within 
each original training example S x is replaced with its corresponding average 
value V 1|8V9 , through V y{ftvgl , where "y" is the number of variables, and the 
modified training example S/ is again executed through the multiple networks 

30 to produce a modified output for each network for each variable. The 

differences between the output from the original training example S x and the 
modified output for each input variable are the squared and summed 
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(accumulated) to obtain individual sums corresponding to each input variable. 
To provide an illustration, for example, for 10 separate neural networks N,-N 10 
and 5 different training examples S r S s , each having 15 variables V,-V 15 , each 
of the 5 training examples would be run through the 10 networks to produce 50 
5 total outputs. Taking variable V } from each of the training examples, an 

average value V 1(avfl> is calculated. This averaged variable V 1(Bvg , is substituted 
into each of the 5 training examples to create modified training examples S/-S 5 ' 
and they are again run through the 10 networks. Fifty modified output values 
are generated by the networks Ni~N 10 for the 5 training examples, the 

10 modification being the result of using the average value variable V 1uvg) . The 
difference between each of the fifty original and modified output values is 
calculated, i.e., the original output from training S 4 in network N 6 : OUT(S 4 N 6 ) is 
subtracted from the modified output from training example S 4 in network N 6 , 
OUT(S 4 'N 6 ). That difference value is squared [OUT(S 4 'N 6 )-OUT(S 4 N 6 )] 2 v ,. This 

15 value is summed with the squared difference values for all combinations of 
networks and training examples for the iteration in which variable V, was 
substituted with its average value V ljnvg) , i.e. , 

20 X $,J OXiT l S * N n )-OUT(S x NX 



25 Next, the process is repeated for variable #2, finding the differences between 
the original and modified outputs for each combination of network and training 
example, squaring, then summing the differences. This process is repeated for 
each variable until all 15 variables have been completed. 

Each of the resultant sums is then normalized so that if all variables 

30 contributed equally to the single resultant output, the normalized value would be 
1 .0. Following the preceding example, the summed squared differences for 
each variable are summed to obtain a total summed squared difference for all 
variables. The value for each variable is divided by the total summed square 
difference to normalize the contribution from each variable. From this 
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information, the normalized value for each variable can be ranked in order of 
importance, with higher relative numbers indicating that the corresponding 
variable has a greater influence on the output. The sensitivity analysis of the 
input variables is used to indicate which variables played the greatest roles in 
5 generating the network output. 

It has been found herein that using consensus networks to perform 
sensitivity analysis improves the variable selection process. For example, if two 
variables are highly correlated, a single neural network trained on the data might 
use only one of the two variables to produce the diagnosis. Since the variables 

10 are highly correlated, little is gained by including both, and the choice of which 
to include is dependent on the initial starting conditions of the network being 
trained. Sensitivity analysis using a single network might show that only one, 
or the other, is important. Sensitivity analysis derived from a consensus of 
multiple networks, each trained using different initial conditions, may reveal that 

15 both of the highly correlated variables are important. By averaging the 

sensitivity analysis over a set of neural networks, a consensus is formed that 
minimizes the effects of the initial conditions. 

Chi-square contingency table 
When dealing with sparse binary data, a positive response on a given 

20 variable might be highly correlated to the condition being diagnosed, but occur 
so infrequently in the training data that the importance of the variable, as 
indicated by the neural network sensitivity analysis, might be very low. In order 
to catch these occurrences, the Chi-square contingency table is used as a 
secondary ranking process. A 2X2 contingency table Chi-square test on the 

25 binary variables, where each cell of the table is the observed frequency for the 
combination of the two variables (Fig. 3, Step F) is performed. A 2X2 
contingency table Chi-square test is performed on the continuous variables 
using optimal thresholds (which might be empirically-determined) (Step G)> The 
binary and continuous variables that have been based on Chi-square analysis are 

30 ranked (Step H). 

The standard Chi-square 2X2 contingency table operative on the binary 
variables (Step F) is used to determine the significance of the relationship 
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between a specific binary input variable and the desired output (as determined 
by comparing the training data with the known single output result). Variables 
that have a low Chi-square value are typically unrelated to the desired output. 
For variables that have continuous values, a 2X2 contingency table can 
5 be constructed (Step G) by comparing the continuous variable to a threshold 
value. The threshold value is modified experimentally to yield the highest 
possible Chi-square value. 

The Chi-square values of the continuous variables and of the binary 
variables can then be combined for common ranking (Step H). A second level 

10 of ranking can then be performed that combines the Chi-square-ranked variables 
with the sensitivity-analysis-ranked variables (Step L). This combining of 
rankings allows variables that are significantly related to the output but that are 
sparse (Le, values that are positive or negative in only a small percentage of 
cases) to be included in the set of important variables. Otherwise, important 

15 information in such a non-linear system could easily be overlooked. 

Selection of important variables from among the ranked variables 
As noted above, important variables are selected from among the 
identified variables. Preferably the selection is effected after ranking the 
variables at which time a second level ranking process is invoked. A method for 

20 identification of important variables (parameters) or sets thereof for use in the 
decision-support systems is also provided. This method, while exemplified 
herein with reference to medical diagnosis, has broad applicability in any field, 
such as financial analysis and other endeavors that involve statistically-based 
prediction, in which important parameters or variables are selected from among 

25 a plurality. 

In particular, a method for selecting effective combinations of variables is 
provided. After (1) providing a set of "n H candidate variables and a set of 
"selected important variables", which initially is empty; and (2) ranking all 
candidate variables based on a chi square and sensitivity analysis, as described 
30 above, the method involves: (3) taking the highest "m" ranked variables one at 
a time, where m is from 1 up to n, and evaluating each by training a consensus 
of neural nets on that variable combined with the current set of important 
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variables; (4) selecting the best of the m variables, where the best variable is 
the one that most improves performance, and if it improves performance, 
adding it to the "selected important variable" set, removing it from the 
candidate set and continuing processing at step (3) otherwise continuing by 
going to step (5); (5) if all variables on the candidate set have been evaluated, 
the process is complete, otherwise continue taking the next highest "m" ranked 
variables one at a time, and evaluating each by training a consensus of neural 
nets on that variable combined with the current set of important selected 
variables and performing step (4). 

In particular, the second level ranking process (Step L) starts by adding 
the highest ranked variable from the sensitivity analysis (Step K) to the set of 
important variables (Step H). Alternatively, the second level ranking process 
could be started with an empty set and then testing the top several (x) variables 
from each of the two sets of ranking. This second level ranking process uses 
the network training procedure (Step I) on a currently selected partition or 
subset of variables from the available data to train a set of neural networks. 
The ranking process is a network training procedure using the current set of 
"important" variables (which generally will initially be empty) plus the current 
variable being ranked or tested for ranking, and uses a greedy algorithm to 
optimize the set of input variables by myopically optimizing the input set based 
upon the previously identified important variable(s), to identify the remaining 
variable(s) which improve the output the most. 

This training process is illustrated in Fig. 4. The number of inputs used 
by the neural network is controlled by excluding inputs which are found to not 
contribute significantly to the desired output, i.e., the known target- output of 
the training data. A commercial computer program, such as ThinksPro™ neural 
networks for Windows™ (or TrainDos* the DOS version) by Logical Designs 
Consulting, Inc, La Jolla, California, or any other such program that one of skill 
in the art can develop may be used to vary the inputs and train the networks. 

A number of other commercially available neural network computer 
programs may be used to perform any of the above operations, including 
Brainmaker ,m , which is available from California Scientific Software Co., Nevada 
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Adaptive Solutions, Beaverton, OR; Neural Network Utility/2 tm , from 
NeuralWare, Inc., Pittsburgh, PA; NeuroShelP™ and NeuroWindows tm , from 
Ward Systems Group, Inc., Frederick, MD. Other types of data mining tools, 
i.e. , decision-support systems, that will provide the function of variable 
5 selection and network optimization may be designed or other commercially 
available systems may be used. For example, NeuroGenetic Optimizer" from 
BioComp Systems, Inc., Redmond, WA; and Neuro Forecaster/GENETICA, from 
New Wave Intelligent Business Systems (NIBS) Pte Ltd., Republic of Singapore, 
use genetic algorithms that are modelled on natural selection to eliminate poor- 

10 performing nodes within network population while passing on the best 

performing rates to offspring nodes to "grow" an optimized network and to 
eliminate input variables which do not contribute significantly to the outcome. 
Networks based on genetic algorithms use mutation to avoid trapping in local 
minima and use crossover processes to introduce new structures into the 

15 population. 

Knowledge discovery in data (KDD) is another data mining tool, decision- 
support system, designed to identify significant relationship is that exist among 
variables, and are useful when there are many possible relationships. A number 
of KDD systems are commercially available including Darwin tm , from Thinking 

20 Machines, Bedford, MA; Mineset tm , from Silicon Graphics, Mountain View, CA, 
and Eikoplex tm from Ultragem Data Mining Company, San Francisco, CA. 
(Eikoplex !m has been used to provide classification rules for determining the 
probability of the presence of heart disease.) Others may be developed by 
those of skill in the art. 

25 Proceeding with the ranking procedure, if, for example, x is set to 2, 

then the top two variables from each of the two ranking sets will be tested by 
the process <Fig. 3, Steps L, S), and results are checked to see if the test 
results show improvement (Step T). If there is an improvement, the single best 
performing variable is added to the set of "important" variables, and then that 

30 variable is removed from the two rankings (Fig. 3, Step U) for further testing 

(Step S). If there is no improvement, then the process is repeated with the next 
x variables from each set until an improvement is found or all of the variables 
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from the two sets have been tested. This process is repeated until either the 
source sets are empty, i.e., aii relevant or important variables have been 
included in the final network, or all of the remaining variables in the sets being 
tested are found to be below the performance of the current list of important 
5 variables. This process of elimination greatly reduces the number of subsets of 
the available variables which must be tested in order to determine the set of 
important variables. Even in the worst case, with ten available variables, the 
process would test only 34 subsets where x = 2 and only 19 subsets of the 
1024 possible combinations if x = 1 . Thus, where there are 100 available 
10 variables, only 394 subsets would be tested where x = 2. The variables from 
the network with the best test performance are thus identified for use (Fig. 3, 
Step V). 

Then the final set of networks is trained to perform the diagnosis (Fig. 4, 
Steps M, N, Q, R). Typically, a number of final neural networks are trained to 

15 perform the diagnosis. It is this set of neural networks (a that can form the 

basis of a deliverable product to the end user. Since different initial conditions 
(initial weights) can produce differing outputs for a given network, it is useful to 
seek a consensus. (The different initial weights are used to avoid error from 
trapping in local minima.) The consensus is formed by averaging the outputs of 

20 each of the trained networks which then becomes the single output of the 
diagnostic test. 

Training a consensus of networks 
Fig. 4 illustrates the procedure for the training of a consensus of neural 
networks. It is first determined whether the current training cycle is the final 

25 training step (Step M). If yes, then all available data are placed into the training 
data set (i.e., P= 1) (Step N). If no, then the available data are divided into P 
equal-sized partitions, randomly selecting the data for each partition (Step O). 
In an exemplary embodiment, for example five partitions, e.g. , P^P 5 , are created 
from the full set of available training data. Then two constructions are 

30 undertaken (Step P). First, one or more of the partitions are copied to a test file 
and the remaining partitions are copied to a training file. Continuing the 
exemplary embodiment of five partitions, one of the partitions, e.g., P,,, 
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representing 20% of the total data set, is copied to the test file. The remaining 
four files, P 2 -P 4 , are identified as training data. A group of N neural networks is 
trained using the training partitions, each network having different starting 
weights (Step Q). Thus, in the exemplary embodiment, there will be 20 
5 networks (N = 20) with starting weights selected randomly using 20 different 
random number seeds. Following completion of training for each of the 20 
networks, the output values of all 20 networks are averaged to provide the 
average performance on the test data for the trained networks. The data in the 
test file (partition P,} is then run through the trained networks to provide an 

10 estimate of the performance of the trained networks. The performance is 

typically determined as the mean squared error of prediction, or misclassification 
rate. A final performance estimate is generated by averaging the individual 
performance estimates of each network to produce a completed consensus 
network (Step R). This method of training by partitioning the available data into 

15 a plurality of subsets is generally referred to as the "holdout method" of 

training. The holdout method is particularly useful when the data available for 
network training is limited. 

Test set performance can be empirically maximized by performing 
various experiments that identify network parameters that maximize test set 

20 performance. The parameters that can be modified in this set of experiments 

are 1) the number of hidden processing elements, 2) the amount of noise added 
to the inputs, 3) the amount of error tolerance, 4) the choice of learning 
algorithm, 5) the amount of weights decay, and 6) the number of variables. A 
complete search of all possible combinations is typically not practical, due to 

25 the amount of processing time that is required. Accordingly, test networks are 
trained with training parameters chosen empirically via a computer program, 
such as ThinksPro™ or a user developed program, or from the results of existing 
test results generated by others who are working in the field of interest. Once a 
"best" configuration is determined, a final set of networks can be trained on the 

30 complete data set. 
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3. Development of biochemical diagnostic test 

A similar technique for isolating variables may be used to build or 
validate a biochemical diagnostic test, and also to combine a biochemical 
diagnostic test data with the patient history diagnostic test to enhance the 
5 reliability of a medical diagnosis. 

The selected biochemical test can include any test from which useful 
diagnostic information may be obtained in association with a patient and/or 
patient's condition. The test can be instrument or non-instrument based and 
can include the analysis of a biological specimen, a patient symptom, a patient 

10 indication, a patient status, and/or any change in these factors. Any of a 

number of analytical methods can be employed and can include, but are not 
limited to, immunoassays, bioassays, chromatography, monitors, and imagers. 
The analysis can assess analytes, serum markers, antibodies, and the like 
obtained from the patient through a sample. Further, information concerning 

15 the patient can be supplied in conjunction with the test. Such information 

includes, but is not limited to, age, weight, blood pressure, genetic history, and 
the other such parameters or variables. 

The exemplary biochemical test developed in this embodiment employs a 
standardized test format, such as the Enzyme Linked Immunosorbent Assay or 

20 ELISA test, although the information provided herein may apply to the 

development of other biochemical or diagnostic tests and is not limited to the 
development of an ELISA test (see, e.g. . Molecular Immunology: A Textbook , 
edited by Atassi et aL Marcel Dekker Inc., New York and Basel 1984, for a 
description of ELISA tests). Information important to the development of the 

25 ELISA test can be found in the Western Blot test, a test format that determines 
antibody reactivity to proteins in order to characterize antibody profiles and 
extract their properties. 

A Western Blot is a technique used to identify, for example, particular 
antigens in a mixture by separating these antigens on polyacrylamide gels, 

30 blotting onto nitrocellulose, and detecting with labeled antibodies as probes. 
(See, for example, Basic and Clinical Immunology , Seventh Edition, edited by 
Stites and Terr, Appleton and Large 1991, for information on Western Blots.) It 
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is, however, sometimes undesirable to employ the Western Blot test as a 
diagnostic tool. If instead, ranges of molecular weight that contain relevant 
information to the diagnosis can be pre-identified then this information can be 
"coded" into an equivalent ELISA test. 
5 In this example, the development of an effective biochemical diagnostic 

test is dependent upon the availability of Western Blot data for the patients for 
which the disease condition is known or suspected. Referring to Fig. 5, 
Western Blot data are used as a source (Step W), and the first step in 
processing the Western Blot data are to pre-process the Western Blot data for 

10 use by the neural network (Step X). Images are digitized and converted to fixed 
dimension training records by using a computer to perform the spline 
interpolation and image normalization. It is necessary to align images on a given 
gel based only on information in the image in order to use data from multiple 
Western Blot tests. Each input of a neural network needs to represent a 

15 specific molecular weight or range of molecular weights accurately. Normally, 
each gel produced contains a standards image for calibration, wherein the 
proteins contained are of a known molecular weight, so that the standards 
image can also be used for alignment of images contained within the same 
Western Blot. For example, a standard curve can be used to estimate the 

20 molecular weight range of other images on the same Western Blot and thereby 
align the nitrocellulose strips. 

The process for alignment of images is cubic spline interpolation. This is 
a method which guarantees smooth transitions at the data points represented 
by the standards. To avoid possible performance problems due to extrapolation, 

25 termination conditions are set so that extrapolation is linear. This alignment 
step minimizes the variations in the estimates of molecular weight for a given 
band on the output of the Western Blot. 

The resultant scanned image is then processed to normalize the density 
of the image by scaling the density so that the darkest band has a scaled 

30 density of 1 .0 and the lightest band is scaled to 0.0. The image is then 

processed into a fixed length vector of numbers which become the inputs to a 
neural network, which at the outset must be trained as hereinafter explained. 
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A training example is built in a process similar to that previously 
described where the results generated from the processing of the Western Blot 
data are trained (Step Y). To minimize the recognized problems of dependency 
on starting weights, redundancy among interdependent variables, and 
5 desensitivity resulting from overtraining a network, it is helpful to train a set of 
neural networks (consensus) on the data by the partitioning method discussed 
previously. 

From the sensitivity analysis of the training runs on the processed 
Western Biot data, regions of significantly contributing molecular weights (MW) 

10 can be determined and identified (Step AA). As part of the isolation step, 

inputs in contiguous regions are preferably combined into "bins" as long as the 
sign of the correlation between the input and the desired output is the same. 
This process reduces the typical 100-plus inputs produced by the Western Blot, 
plus the other inputs, to a much more manageable number of inputs of less than 

15 about twenty. 

In a particular embodiment, it may be found that several ranges of 
molecular weight may correlate with the desired output, indicative of the 
condition being diagnosed. A correlation may be either positive or negative. A 
reduced input representation may be produced by using a Gaussian region 

20 centered on each of the peaks found in the Western Blot training, with a 

standard deviation determined so that the value of the Gaussian was below 0.5 
at the edges of the region. 

In a specific embodiment, the basic operation to generate the neural 
network input is to perform a convolution between the Gaussian and the 

25 Western Blot image, using the log of the molecular weight for calculation. 

The data may be tested using the holdout method, as previously 
described. For example, five partitions might be used where, in each partition, 
80% of the data are used for training and 20% of the data are used for testing. 
The data are shuffled so that each of the partitions is likely to have examples 

30 from each of the gels. 

Once the molecular weight regions important to diagnosis have been 
identified (Step AA), one or more tests for the selected region or regions of 
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molecular weight may be built (Step AB). The ELISA biochemical test is one 
example. The selected region or regions of molecular weight identified as 
important to the diagnosis may then be physically identified and used as a 
component of the ELISA biochemical test. Whereas regions of the same 
5 correlation sign may, or may not, be combined into a single ELISA test, regions 
of differing correlation signs should not be combined into a single test. The 
value of such a biochemical test may then be determined by comparing the 
biochemical test result with the known or suspected medical condition. 

In this example, the development of a biochemical diagnostic test may be 

10 enhanced by combining patient data and biochemical data in a process shown in 
Fig. 2. Under these conditions, the patient history diagnostic test is the basis 
for the biochemical diagnostic test. As explained herein, the variables that are 
identified as important variables are combined with data derived from the 
Western Blot data in order to train a set of neural networks to be used to 

15 identify molecular weight regions that are important to a diagnosis. 

Referring to Fig. 2, Western Blot data are used as a source (Step W) and 
pre-processed for use by the neural network as described previously (Step X). 
A training example is built in a process similar to that previously described 
wherein the important variables from the patient history data and the results 

20 generated from the processing of the Western Blot data are combined and are 
trained using the combined data (Step Y). In parallel, networks are trained on 
patient history data, as described above (Step Z). 

To minimize the recognized problems of dependency on starting weights, 
redundancy among interdependent variables, and desensitivity resulting from 

25 overtraining a network, it was found that it was preferable to train a set of 

neural networks (consensus set) on the data by the partitioning method. From 
the sensitivity analysis of the training runs on patient history data alone and on 
combined data, regions of significantly contributing molecular weights can be 
determined and identified as previously described (Step AA). As a further step 

30 in the isolation process, a set of networks is thereafter trained using as inputs 
the combined patient history and bin information in order to isolate the 
important bins for the Western Blot data. The "important bins" represent the 
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important regions of molecular weight related to the diagnosis considering the 
contribution of patient history information. These bins are either positively or 
negatively correlated with the desired output of the diagnosis. 

Once the molecular weight regions important to diagnosis have been 
5 identified (Step AA), one or more tests for the selected region or regions may be 
built and validated as previously described {Step AB). The designed ELISA tests 
are then produced and used to generate ELISA data for each patient in the 
database (Step AC). Using ELISA data and the important patient history data as 
input, a set of networks is trained using the partition approach as described 

10 above (Step AE). The partition approach can be used to obtain an estimate of 
the lower bound of the biochemical test. The final training (Step AE) of a set of 
networks, i.e., the networks to be used as a deliverable product, is made using 
all available data as part of the training data. If desired, new data may be used 
to validate the performance of the diagnostic test (Step AF). The performance 

15 on all the training data becomes the upper bound on the performance estimate 
for the biochemical test. The consensus of the networks represents the 
intended diagnostic test output (AG). This final set of neural networks can then 
be used for diagnosis. 

4. Improvement of neural network performance 

20 An important feature of the decision-support systems, as exemplified 

with the neural networks, and methods provided herein is the ability to improve 
performance. The training methodology outlined above may be repeated as 
more information becomes available. During operation, all input and output 
variables are recorded and augment the training data in future training sessions. 

25 In this way, the diagnostic neural network may adapt to individual populations 
and to gradual changes in population characteristics. 

If the trained neural network is contained within an apparatus that allows 
the user to enter the required information and outputs to the user the neural 
network score, then the process of improving performance through use may be 

30 automated. Each entry and corresponding output is retained in memory. Since 
the steps for retraining the network can be encoded into the apparatus, the 
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network can be re-trained at any time with data that are specific to the 
population. 

5. Method for evaluating the effectiveness of a diagnostic test course of 
treatmentt 

5 Typically, the effectiveness or usefulness of a diagnostic test is 

determined by comparing the diagnostic test result with the patient medical 
condition that is either known or suspected. A diagnostic test is considered to 
be of value if there is good correlation between the diagnostic test result and 
the patient medical condition; the better the correlation between the diagnostic 

10 test result and the patient medical condition, the higher the value placed on the 
effectiveness of the diagnostic test. In the absence of such a correlation, a 
diagnostic test is considered to be of lesser value. The systems provided 
herein, provide a means to assess the effectiveness of a biochemical test by 
determining whether the variable that corresponds to that test is an important 

15 selected variable. Any test that yields data that improves the performance of 
the system is identified. 

A method by which the effectiveness of a diagnostic test may be 
determined, independent of the correlation between the diagnostic test result 
and the patient medical condition (Fig. 6) is described below. A similar method 

20 may be used to assess the effectiveness of a particular treatment. 

In one embodiment, the method compares the performance of a patient 
history diagnostic neural network trained on patient data alone, with the 
performance of a combined neural network trained on the combination of 
patient historical data and biochemical test data, such as ELISA data. Patient 

25 history data are used to isolate important variables for the diagnosis (Step AH), 
and final neural networks are trained (Step AJ), all as previously described. In 
parallel, biochemical test results are provided for all or a subset of the patients 
for whom the patient data are known (Step AK), and a diagnostic neural 
network is trained on the combined patient and biochemical data by first 

30 isolating important variables for the diagnosis (Step AL), and subsequently 
training the final neural networks (Step AM), all as previously described. 
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The performance of the patient history diagnostic neural network derived 
from Step AJ is then compared with the performance of the combined 
diagnostic neural network derived from Step AM, in Step AN. The performance 
of a diagnostic neural network may be measured by any number of means. In 
one example, the correlations between each diagnostic neural network output to 
the known or suspected medical condition of the patient are compared. 
Performance can then be measured as a function of this correlation. There are 
many other ways to measure performance. In this example, any increase in the 
performance of the diagnostic neural network derived from Step AM over that 
derived from Step AJ is used as a measure of the effectiveness of the 
biochemical test. 

A biochemical test in this example, and any diagnostic test in general, 
that lacks sufficient correlation between that test result and the known or 
suspected medical condition, is traditionally considered to be of limited utility. 
Such a test may be shown to have some use through the method described 
above, thereby enhancing the effectiveness of that test which otherwise might 
be considered uninformative. The method described herein serves two 
functions: it provides a means of evaluating the usefulness of a diagnostic test, 
and also provides a means of enhancing the effectiveness of a diagnostic test. 
6. Application of the methods to identification of variables for diagnosis 

and development of diagnostic tests 

The methods and networks provided herein provide a means to, for 
example, identify important variables, improve upon existing biochemical tests, 
develop new tests, assess therapeutic progress, and identify new disease 
markers. To demonstrate these advantages, the methods have* been to 
pregnancy related events, such as the likelihood of labor and delivery during a 
particular period. 

Predicting pregnancy related events, such as the likelihood of delivery 
within a particular time period 

The methods herein may be applied to any disorder or condition, and are 
particularly suitable for conditions in which no diagnostic test can be adequately 
correlated or for which no biochemical test or convenient biochemical test is 
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available. The methods herein have been applied to predicting pregnancy 

related events, such as the likelihood of delivery within a particular time period. 

Biochemical and other markers for assessment of the risk of 
preterm delivery or delivery within a selected period of time 

5 Determination of impending birth is of importance, example, for 

increasing neonatal survival of infants born before 34 weeks. The presence of 

fetal fibronectin in secretion samples from the vaginal cavity or the cervical 

canal from a pregnant patient after week 20 of pregnancy is associated with a 

risk of labor and delivery before 34 weeks. Methods and kits for screening for 

10 fetal fibronectin in body fluids and tissues, particularly in secretion samples from 
the vaginal cavity or the cervical canal, of a pregnant patient after week 20 of 
pregnancy are available (see, U.S. Patent Nos. 5,516,702, 5,468,619, and 
5,281,522, and 5,096,830; see, also U.S. Patent Nos. 5,236,846, 5,223,440, 
5,185,270; see also, U.S. Patent Nos. 5,623,939, 5,480,776, 5,474,927, 

15 5,279,941 and 5,091,170 for other biochemical tests and markers). 

For example U.S. Patent No. 5,468,619 provides a biochemical 
indication of increased risk of impending delivery. The method is particularly 
useful to identify those pregnant women who are at increased risk for preterm 
delivery. The method can also be used to detect those women who are at risk 

20 for post-date delivery. The method includes obtaining a cervicovaginal secretion 
sample from a pregnant patient after about week twelve of gestation and 
determining the level of total fibronectin in the sample. The presence of an 
elevated fibronectin level in the sample indicates an increased risk of imminent 
delivery. The test is a sensitive and specific screen for pregnancies at risk and 

25 can detect impending delivery about two to three weeks prior to delivery. 

This test is preferably administered to women at about 12 weeks 
gestation and repeated at each perinatal visit (every two to four weeks) until at 
least week 37, preferably until delivery, if the test is negative. For those 
patients whose assay result indicates an increased risk of preterm delivery, a 

30 test of the patient's fetal fibronectin level can be made to confirm the increased 
risk and to estimate time of delivery. Using these results in combination 
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with a decision support system or a part of the set of training variable improves 
the predictive ability of the biochemical test. 

U.S. Patent No. 5,281,522 provides a biochemical test for increased risk 
of preterm labor and rupture of the amniotic membrane after week 20 of 
5 pregnancy is directed to an assay of a test sample removed from the vicinity of 
the posterior fornix, cervical canal, or cervical os. 

U.S. Patent No. 5,516,702 describes a biochemical indication and 
method of asssessing increased risk of impending preterm delivery. The method 
involves obtaining a cervicovaginal secretion sample from a pregnant patient 

10 after and determining the level of a local inflammatory product protein, such as 
IL-6, in the sample. The presence of an elevated level of the selected local 
inflammatory product in the sample indicates an increased risk of imminent 
deliver/. The test is a sensitive and specific screen for pregnancies at risk and 
can detect impending delivery as early as two to three weeks prior to delivery. 

15 For those patients whose assay result indicates an increased risk of preterm 
delivery, a test of the patient's fetal fibronectin level can be made to confirm 
the increased risk and to estimate how soon the delivery may be. In addition, 
those patients can be carefully monitored, as for other patients at risk. 

U.S. Patent No. 5,480,776 describes a method for predicting the onset 

20 of preterm labor at 36 weeks or earlier by analyzing unconjugated estriol levels 
in a body fluid, saliva, and correlating the levels with either: 

a predetermined standard unconjugated estriol concentration for the body fluid, 
or a previously measured unconjugated estriol concentration in the body fluid of 
said pregnant human to determine a rate of increase in unconjugated estriol 

25 concentration in the body fluid of said pregnant human. A higher concentration 
of unconjugated estriol in the body fluid of the pregnant human relative to a 
predetermined standard unconjugated estriol concentration, or an elevated rate 
of increase in unconjugated estriol concentration in the body fluid is an 
indication of potential onset of preterm labor. 

30 Estetrol is also used as an indicator of preterm delivery (see, International 

PCT application No. WO96/03929, entitled Method for Prediction of Premature 
Delivery Using Estetrol (E 4 ) As an Indicator. 
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Other markers indicative of assessment of a risk of preterm delivery or 
delivery within a selected period of time that be used in combination with the 
methods and decision support systems provided herein include, but are not 
limited to the following: corticotropin-releasing hormone (CRH), which can be 
5 sampled, for example, from serum; estriol E3 and estretol, noted above, which 
can be sampled, for example, from saliva or serum; cerivcovaginal 
dehydroepiandrosterone (DHEA), which can be sampled, for example, from 
serum; FasL (ligand for Fas receptor, an oncogene that mediates programmed 
cell death), which can be sampled, for example, from serum; beta human 

10 chorionic gonadotropin (/?-hcG), which can be sampled, for example, from serum 
and cerivcovaginal area; insulin-like growth factor binding protein-1 (IGFBP-1), 
which can be sampled, for example, from serum; Uterine Artery Doppler, which 
can be sampled, for example, from uterus (by transvaginal ultrasonography 
(TVS)); Umbilical Artery Doppler, which can be sampled, for example, from 

15 fetus; Mid. Cerebral Artery Doppler, which can be sampled, for example, from 
fetus; Ultrasound estimated fetal weight percentile, which can be sampled, for 
example, from fetus; IL-6, which can be sampled, for example, from serum, 
cervix; GCSF, which can be sampled, for example, from serum 
bacterial vaginosis, which can be sampled, for example, from vagina 

20 gross or occult blood, which can be sampled, for example, from vagina; tPA 
(activity), which can be sampled, for example, from plasma; 
thrombin-ATIII Complexes, which can be sampled, for example, from plasma; 
amniotic fluid index, for example, from uterus no. fetuses, which can be 
sampled, for example, from uterus; matrix metalloprteinase-1 (MMP-1), which 

25 can be sampled, for example, from cervix; matrix metalloprteinase-9 (MMP-9), 
which can be sampled, for example, from cervix; fFN, which can be sampled, 
for example, from cervix and vagina cervical length (TVS), which can be 
sampled, for example, from vagina. 

As noted above, methods and kits for screening for fetal fibronectin in 

30 body fluids and tissues, particularly in secretion samples from the vaginal cavity 
or the cervical canal of a pregnant patient, particularly after week 20 of 
pregnancy are available (see, U.S. Patent Nos. 5,516,702, 5,468,619, and 



WO 99/09507 



PCT/US98/16891 



-44- 

5,281,522, and 5,096,830). The correlation between the presence of fetal 
fibronectin in these secretions and the labor and delivery before 34-35 weeks is 
not perfect; there are significant false-negative and false-positive rates. 
Consequently, to address the need for methods to assess the likelihood of labor 
5 and delivery before 34 weeks and to improve the predictability of the available 
tests, the methods herein have been applied to development of a decision- 
support system that assesses the likelihood of certain pregnancy related events. 
In particular, neural nets for predicting delivery before (and after) 34 weeks of 
gestation have been developed. Neural networks and other decision-support 

10 systems developed as described herein can improve the performance of the fetal 
fibronectin (fFN) test by lowering the number of false positives. 

The results, which are shown in the EXAMPLE, demonstrate that use of 
the methods herein can improve the diagnostic utility of existing tests by 
improving predictive performance. EXEMPLARY neural networks and 

15 implementing software are also described. 

In addition, the methods herein can identify additional markers and tests 
of relevance or use in assessing the risk of preterm delivery or delivery within a 
selected period of time. 

PreTerm Delivery Risk Assessment Software 

20 The Pre-term Delivery Risk Assessment Software program provides a , 

means to input patient historical information and fFN test results into a database 
of fixed length ASCII records, and to perform the calculations necessary to 
generate inputs to three neural network tests used to evaluate the patients risks 
related to pre-term delivery. The software generates outputs that define the 

25 risk of preterm delivery. The Preterm Delivery Risk Assessment Software 

provided herein classifies the fFN ELISA positive results into 3 clinically distinct 
groups. In so doing, more than 50% of the fFN ELISA false positive results can 
be immediately identified. Moreover, about 35% of the true positive results can 
be rescued. The combination of the Preterm Delivery Risk Assessment 

30 Software with the ELISA test result provides new information which the 
clinician can use to improve the management of symptomatic patients. In 
particular, risk of delivery less than or equal to 34 week, 6 days, risk of delivery 
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less than or equal to 7 days from time of sampling for fFN, and risk of delivery 
less than or equal to 14 days from time of sampling for fFN. The exemplified 
software uses neural networks designated EGA6, EGA7f and EGA14f (see 
Example) herein, but can be used with any nets provided herein or developed 
5 based on the methods provided herein. 

The following is a description of the operation, inputs and outputs of the 
software. 

A. User Interface 

A typical user interface is depicted in FIGURES 7-10 and exemplary 
10 printed outputs are depicted in FIGURES 11A and 1 1 B. 

Main Menu 

The main menu, tool bar and results display appear as shown in FIGURE 
7. The various fields in the main window are calculated as follows: 

File: The name of the fdb file opened by the user. 
15 Current Record: The record number counting from 1 that is currently 

displayed. 

Number of records: The count of records contained in the open file. 

Lab ID #: The contents of the associated field in the fixed length data 
record entered by the user. 
20 Patient Name: The first, middle initial, and last name of the patient from 

the fixed length data 
record. 

Pre-term Delivery Risk < 34 weeks 6 days, which is the consensus 
score from the ega6 set of neural networks. 
25 Pre-term Delivery Risk < 7 days: The consensus score from the egad7f 

set of neural networks. 

Pre-term Delivery Risk < 14 days: The consensus score from the 
egad14f set of neural networks. 

The File item on the main menu contains the following sub menu items 
30 and functions; 

Open: Open a fixed length database file (.fdb) for entry and examination. 
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Print: Print the current record in one of the two formats as specified in 
the Options menu. 

Print Setup: Provides standard Windows support for print functions 

setup. 

5 Print Preview: Provides standard Windows support for print viewing. 

MRU List: Provides a list of the four Most Recently Used files. 
Exit: To Exit the program. 

The Record item on the main menu will contain the following sub menu 
items and functions: 
10 First Record: Display the first record in the database file. 

Next Record: Display the next record in the database file. 

Previous Record: Display the previous record in the database file. 

Last Record: Display the last record in the database file. 

Go to Record: Opens a dialog to go to a specific record number or search 
15 for a specific Lab ID #. 

Edit Record: Opens a dialog to allow the examination and modification of 
data in the displayed 
record. 

New Record: Creates a new record at the end of the database and 
20 automatically edits the record. 

The Options item on the main menu will contain the following sub menu 
items and functions; 

Print full form: When checked, the print function will print the full record 
as shown in the edit 

25 record dialog box. When unchecked, the print function will print the information 
shown in the main window. The default setting is unchecked. 

Clear sub fields: When checked, sub fields will be cleared when field is 
unchecked in the edit dialog. The default setting is checked. 

The View item on the main menu will contain the following sub menu 
30 items and functions: 

Toolbar: Standard Windows Toolbar appears when checked. 
Status Bar: Standard Windows Status Bar appears when checked. 
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The Help item on the main menu will contain the following sub menu items and 
functions; 

About PTDinp: Provide version number, program icon, and developer of 
the program. 

5 Tool Bar buttons will be provided for the following functions: 
File Open 
View First Record 
View Previous Record 
View Next Record 
10 View Last Record 

Edit Record 
New Record 
Go To Record 
Help About 
1 5 Edit Dialog 

An exemplary Edit Record dialog box is set forth in FIGURE 8. Through 
this dialog the user can exam, change or input patient specific data into a fixed 
length database record. The table below provides the size and location of each 
item in the fixed length database record. For entry into the dialog box relevant 
20 items are checked; all checked items are assigned a value of 1 , all others are 

assigned a value of 0. The alphanumeric fields in the dialog box, such as Lab ID 

name, date of birth, EGA boxes, G (gravity), P (parity), A (abortions) are 
assigned the entered values. The table set forth (True = checked, false = 
unchecked) below summarizes how the information entered into the dialog box 
25 is converted for storage in the fixed length database record. . 



NAME 


POSITION 


WIDTH 


DESCRIPTION 


LAB ID n 


1 


12 


ACSII text 


LAST NAME 


13 


24 


ACSII text 


FIRST NAME 


37 


24 


ACSII text 


MIDDLE INITIAL 


61 


2 


ACSII text 


DATE OF BIRTH 


93 


10 


ACSII mm/dd/w 
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NAME 


POSITION 


WIDTH 


DESCRIPTION 




ETHNIC ORIGIN WHITE 


103 


2 


0« FALSE 1 = TRUE 




ETHNIC ORIGIN BLACK 


105 


2 


0- FALSE 1 «= TRUE 




FTHNIT ORIGIN A^IAKJ 


1 07 


2 


n - CAI CC 1 TO 1 1 C 

v — rMwt l — 1 nUc 


5 




1 HQ 




U = rMLot 1 = 1 HUb 




CTUIM!^ HPI^IM MATtWC AMCQirAM 
C 1 nOJJL. UnlVjIlN IMA 1 IVC M rvlCnlLrMiN 


111 

111 


2 


0 o FALSE 1 =TRUE 




FTHNIP omniM OTHFR 


1 1 o 


« 


A — C A 1 CC i _TDl IE 




MARITA1 C.TATIJC. 
IVIM n 1 1 ML Olnl U9 


1 1 R 


4. 


1 = Single (only one box can be 
checked) 










2 = Married 


10 









3 = Divorced 










= widowed 










5 - Living with partner 










6 = Other 




Q v m nt n m <5 nf PrAfnrm lohfir 


117 




\J INO I ** t es 


15 


vay\\ia\ Diuouiiiy 


119 


2 


0 = N/A (check if sub field checked) 










1 = Trace 2 = Medium 3 = Gross 




1 1 ta rjnp Pnntractinn* 

VlOf II ID \>UI III uvllUlld 


121 


2 


O — FAI C,C 1 — TRIJF 
\J — rnLJC I — I nut 




Intermittent lower abdominal pain, 
dull low back pain 


123 


2 


0 = FALSE 1 = TRUE 


20 


Bleeding during the second or third 
trimester 


125 


2 


0 = FALSE 1 = TRUE 




Menstrual-like or intestinal cramping 


127 


2 


0 o FALSE 1 = TRUE 




Change in vaginal discharge 


129 


2 


0- FALSE 1 - TRUE | 


25 


Patient is not "feeling right" 


131 


2 


0 = FALSE 1 =TRUE 




Number/hr. 


133 


2 


0 = Uterine Contractions FALSE j 










1 ="<1" 2 = "1-3" 3 = w 4-6" 4="7-9" 
5«"10-12- 6=">12" 




EGA by SONO 


135 


8 


ACSII weeks. days format U 




EGA by LMP 


143 


8 


ACSII weeks. days format || 


30 


EGA at Sampling 


151 


8 


ACSII weeks. days format || 
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NAME 


POSITION 
ruoi i iuim 


WIDTH 


DESCRIPTION J 


onAVl IT |U . ) 


159 


2 


ASCII number j 


DAD 1 TV ID>I 

rAnl 1 Y \r.) 


161 


2 


ASCII number | 


ADADTIAMC / A .\ 

AoUnllQNb (A.) 


1 63 


2 


ASCII number j 


Number of Preterm delivery 


1 65 


2 


0 = NONE 1 = " 1 " 2 = "2" 3 = " > 2" | 


No previous pregnancies 


1 67 


2 


1 « "Gravity = 0" 


Previous pregnancies with no 
complications 


1 69 


2 


0 - FALSE 1 =TRUE 


History of Preterm delivery 


171 


2 


0 = FALSE 1 = TRUE 


History of preterm PROM 


173 


2 


0 — FALSE 1 = TRUE 


History of incompetent cervix 


175 


2 


0- FALSE 1 = TRUE 


History of PIH/preectampsia 


177 


2 


0 — FALSE 1 =s TRUE 


History of SAB prior to 20 weeks 


179 


2 


0 = FALSE 1 = TRUE 


Multiple Gestation 


1B1 


2 


0 = NONE (unchecked) 








1 « "Twins" 2 «= "Triplets" 3-=" Quads" 


Uterine or cervical abnormality 


183 


2 


0 = FALSE 1 - TRUE 


Cerclage 


185 


2 


0 = FALSE 1 ■ TRUE 


Gestational Diabetes 


187 


2 


0» FALSE 1 -TRUE 


Hypertensive Disorders 


189 


2 


0-FALSE 1 °TRUE 


Dilation 


191 


2 


0 = Unk. Or None checked 








1 = " < 1 - 2 = " 1 " 3 = "1-2" 4= "2" 
D Z-J b = o / = >o 


Cervical Consistency 


193 


2 


blank -(unchecked 








1"Firm" 2 = "Mod" 3 - "Soft" 


Antibiotics 


195 


2 


0 = FALSE 1 =TRUE 


Corticosteroids 


197 


2 


0 = FALSE 1 = TRUE 


Tocolytis 


199 


2 


0 = FALSE 1 = TRUE 


Insulin 


201 


2 


0 — FALSE 1 «=TRUE 


Antihypertensives 


203 


2 


0 — FALSE 1 - TRUE 


Medication: None 


205 


2 


0- FALSE 1 - TRUE 


Medication: Unknown 


207 


2 


0 = FALSE 1 =TRUE 



10 



15 



20 



25 




WO 99/09507 PCT7US98/1 689 1 



-50- 



| NAME 


POSITION 


WIDTH 


DESCRIPTION 


1 Qualitative fFN Result 


209 


2 


0- FALSE 1 «TRUE 


1 <34.6 Net Output Positive 


21 1 


20 


ASCII coded float 


1 <34.6 Net Output Negative 


231 


20 


ASCII coded float 


| < 7 Day Net Output Positive 


251 


20 


ASCII coded float 


( <7 Day Net Output Negative 


271 


20 


ASCII coded float 


| < 14 Day Net Output Positive 


291 


20 


ASCII coded float 


[ < 14 Day Net Output Negative 


311 


20 


ASCII coded float 



Go To Dialog 

10 The Go To dialog box is shown in FIGURE 9. The user may enter either the 

record number or the Lab ID number. When OK is pressed the record is found and 
displayed based on the information contained in a database record. 
Help About Dialog 

The Help About dialog box, which can provide information, such as the title of 
15 the software, version and copyright information, is shown in FIGURE 10. 
B. Pre-term Delivery Risk Evaluation 

1 . Loading the Networks 
When a new database is opened or the program is first run, the neural networks 
associated with the risk evaluations are loaded. For each risk evaluation there are 8 
20 neural networks that must be loaded. This is performed by repeated calls to the 

LoadNet function of the ThinksPro TKSDLL.DLL (a WINDOWS" dynamic link library). 
Other suitable programs can be used to run the neural networks described herein. The 
LoadNet function automatically loads the weights associated with each network. 

For the <> 34 weeks, 6 days evaluation the following nets (described in the 
25 Example) are loaded. 
Ega6_0 
Ega6_1 
Ega6_2 
Ega6_3 
30 Ega6_4 
Ega6_5 
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Ega6_6 
Ega6_7. 

For the < 7 days evaluation the following nets are loaded: 
Egad7fO 
5 Egad7f1 
Egad7f2 
Egad7f3 
Egad7f4 
Egad7f5 
10 Egad7f6 
Egad7f7 

For the < 14 days evaluation the following nets are loaded: 
Egad14f0 
Egad14f1 
15 Egad14f2 
Egadl4f3 
Egad14f4 
Egad14f5 
Egad14f6 
20 Egad14f7 

2. Processing the Inputs and Outputs 
To run the evaluation of the pre-term delivery risks, data from the database 
record must be processed for use by the neural networks. The networks are run for a 
given evaluation when the "calculate risk" button is pressed in the edit record dialog 
25 (FIGURE 8). The positive outputs (described below) of each network are averaged 

together to produce the value that is displayed, printed and placed in the database. The 
negative outputs (described below) are averaged and the result is placed in the 
database only. 
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a. For the <, 34 weeks, 6 days (referred to herein as 34.6) 
evaluation 

The ^ 34.6 networks use 11 inputs generated from the database record. These 
nputs are calculated as follows. 

1. Ethnic Origin White: 1.0 input if TRUE, 0.0 input if FALSE. 

2. Marital Status Living with Partner: 1 .0 input if TRUE, 0.0 
input if FALSE. 

3. EGA by SONO: Convert from weeks. days to weeks. 

4. Vail = EGA by LMP: Convert from weeks. days to weeks. 
Val2 = EGA by SONO: Convert from weeks. days to weeks. If 
Val2 < = 13.0 then input is Val2; Else if the difference between 
Val1 and Val2 is > 2 then input is Val1 . Else input is Val2. 

5. EGA at Sample: Convert from weeks. days to weeks. 

6. If Dilatation none then input is 0.0. 

If Dilatation < 1 then input is 0.0. 
If Dilatation 1 then input is 1 .0. 
If Dilatation 1-2 then input is 1.5. 
If Dilatation 2 then input is 2.0. 
If Dilatation 2-3 then input is 2.0. 
If Dilatation 3 then input is 3.0. 
If Dilatation > 3 then input is 3.0. 

7. If Number of Preterm Delivery = 0 then input is 0.0. 
If Number of Preterm Delivery = 1 then input is 1.0. 

If Number of Preterm Delivery = 2 then input is 2.0. 
If Number of Preterm Delivery > 2 then input is 3.0. 

8. Vaginal Bleeding: 1 .0 input if TRUE, 0.0 input if FALSE. 

9. If Cervical Consistency unchecked then input is 1.823197. 
If Cervical Consistency Firm then input is 1.0. 

If Cervical Consistency Mod then input is 2.0. 
If Cervical Consistency Soft then input is 3.0. 

10. Previous pregnancies with no complications: 1.0 input if 
TRUE, 0.0 input if FALSE. 
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1 1 . FFN Result: 1 .0 input if Positive, 0.0 input if negative. 

b. For the < 7 days evaluation 

The <7 day networks use 7 inputs generated from the database record. These 
inputs are calculated as follows. 

1. Ethnic Origin White: 1 .0 input if TRUE, 0.0 input if FALSE. 

2. Uterine Contractions: 1.0 input if TRUE, 0.0 input if FALSE. 

3. Number of Abortions: Convert to float. 

4. Vaginal Bleeding: 1.0 input if TRUE, 0.0 input if FALSE. 

5. If Number/hr unchecked then input 0.0. 
If Number/hr < 1 then input 1 .0. 

If Number/hr 1-3 then input 2.0. 
If Number/hr 4-6 then input 3.0. 
If Number/hr 7-9 then input 4.0. 
If Number/hr 10-12 then input 5.0. 
If Number/hr > 12 then input 6.0. 

6. No previous pregnancies: 1.0 input if TRUE, 0.0 input if 
FALSE. 

7. fFN Result: 1.0 input if Positive, 0.0 input if negative. 

c. For the ^14 days evaluation 

The < 14 day networks use 7 inputs generated from the database 
record. These inputs are calculated as follows. 

1 . Ethnic Origin Native American: 1 .0 input if TRUE, 0.0 input if 
FALSE. 

2. Marital Status Living with Partner: 1.0 input if TRUE, 0.0 
input if FALSE. 

3. Uterine Contractions: 1.0 input if TRUE, 0.0 input if FALSE. 

4. If Dilatation none then input is 0.0. 
If Dilatation <1 then input is 0.0. 
If Dilatation 1 then input is 1 .0. 

If Dilatation 1-2 then input is 1.5. 
If Dilatation 2 then input is 2.0. 
If Dilatation 2-3 then input is 2.0. 
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If Dilatation 3 then input is 3.0. 
If Dilatation > 3 then input is 3.0. 

5. If Number/hr unchecked then input 0.0. 
If Number/hr < 1 then input 1.0. 

If Number/hr 1-3 then input 2,0. 
If Number/hr 4-6 then input 3.0. 
If Number/hr 7-9 then input 4.0. 
If Number/hr 10-12 then input 5.0. 
If Number/hr > 12 then input 6.0. 

6. No previous pregnancies: 1 .0 input if TRUE, 0.0 input if 
FALSE. 

7. FFN Result: 1 .0 input if Positive, 0.0 input if negative. 
3. Print Functions and Output interpretation 

Based on the print full form option (options menu), print the full form if the 
option is checked and the results only if the option is not checked. FIGURES 1 1 A and 
1 1B show exemplary output formats, with the risk indices for each net, which are 
interpreted according to the following tables: 

Risk of Preterm Delivery (Delivery before 34 weeks 6 days gestation) 



Risk Index 


Interpretation 


< .30 


low risk 


> .30 


high risk f 



Risk of Delivery within 14 days of sampling for fFN Qual ELISA. 



J Risk Index 


Interpretation 


I <; 0.10 


low risk 


I 0.10-0.40 


moderate risk 


1 > 0.40 


high risk 
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Risk of Delivery within 7 days of sampling for fFN Qua! ELISA. 



Risk Index 


Interpretation 


< 0.05 


low risk 


0.05 - 0.60 


moderate risk 


> 0.60 


high risk 



D. Software Performance 

As demonstrated below, the Preterm Delivery Risk Assessment Software 
supplements the fFN ELISA results in a clinically useful manner. By combining patient 

10 history and symptom information with the fFN ELISA test results, the software is able 
to more accurately assess the risk of preterm delivery. The data presented above 
suggest that the software is more capable of discriminating those women truly at risk 
for preterm delivery: whereas the fFN ELISA test has relatively high false positive rates 
and low positive predictive value, the software test reduces false positive observations 

15 by over 50% and doubles predictive value of the positive result. The fFN ELISA test 
allowed clinicians to identify those patients not at risk for preterm delivery. Given the 
significant increase in relative risk and the risk classification of the software test, the 
clinician may now identify those women who are at risk for preterm delivery. This 
capability represents a new advance in the clinical management of women who are 

20 experiencing symptoms of preterm labor. 

In particular, the performance of the Preterm Delivery Risk Assessment Software 
has been evaluated on 763 women experiencing at least one of the following symptoms 
of preterm labor: 

1 . Uterine contractions, with or without pain. 

25 2. Intermittent lower abdominal pain, dull, low backache, pelvic pressure. 

3. Bleeding during the second or third trimester. 

4. Menstrual-like or intestinal cramping, with or without diarrhea. 

5. Change in vaginal discharge—amount, color or consistency. 

6. Not "feeling right". 

30 All 763 women were tested for fFN using the Qualitative fFN ELISA test. Based 

solely on the ELISA test, 149 women tested positive for fFN of which only 20 (13.4%) 
delivered within 7 days and 25 (16.8%) delivered within 14 days. 
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The low positive predictive value of the fFN ELISA test is enhanced by the 
Preterm Delivery Risk Assessment Software, which combines the fFN ELISA result with 
other patient information. 

Table 1 compares the performance of the Qualitative fFN ELISA Test with the 
Preterm Delivery Risk Assessment Software Test for predicting delivery before 35 
weeks completed gestation. The number of false positive observations decreased from 
105 to 42, or about 60%. The decrease in false positive results is accompanied by a 
corresponding increase in true negative results, from 584 for the fFN ELISA to 647 for 
the software test. Moreover, a reduction in false negative results was also observed, 
from 30 for the ELISA test to 25 for the software test. Accordingly, the sensitivity and 
the specificity of the ELISA test are augmented by the software from 59.5% to 66.2% 
and from 84.8% to 90.4%, respectively. The positive predictive nearly doubles, 
increasing from 29.5% to 53.9%, and both the odds ratio and relative risk are 
increased substantially. 



MEASURE 


QUAL fFN ELISA 
TEST 


RISK ASSESSMENT 
SOFTWARE TEST 


True Positive 


44 


49 


False Positive 


105 


42 


True Negative 


584 


647 


False Negative 


30 


25 


Sensitivity 


59.5% 


66.2% 


Specificity 


84.8% 


96.3% 


Pos PV 


29.5% 


53.9% 


Neg PV 


95.1% 


96.3% 


Odds Ratio 


8.2 


30.2 


Relative Risk 


6.0 


14.6 



Table 1 . Performance comparison of Qualitative fFN ELISA Test and the Preterm 
Delivery Risk Assessment Software Test relative to risk of preterm delivery before 35 
completed weeks of gestation. The Risk Assessment Software combines fFN ELISA 
Test results with patient history and symptom information to provide a more accurately 
assess risk of preterm delivery (before 35 completed weeks of gestation). 
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Table 2 compares the performance of the two tests relative to risk of preterm 
delivery within 7 days. The largest difference between the two tests is in the reduction 
of false positive test results of the software when compared to the ELISA test. The 
software decreased the number of false positive observations from 129 to 57, or about 
5 56%. Accompanying this decrease in false positive results is the matching increasing in 
true negative results, from 61 1 to 683. The true positive and false negative results 
remained essentially unchanged. The sensitivity and specificity of the software test is 
much higher than the ELISA test. Compare the sensitivity of 91.3% for the software 
with 87.0% for the ELISA, and the specificity of 92.3% for the software with 92.3% 
10 for the ELISA. Furthermore, the software test doubles the positive predictive value, 

increasing form 13.4% to 26.9%. Finally, the odds ratio is quadrupled and the relative 
risk more than tripled by the software. 



MEASURE 


QUAL fFN ELISA TEST 


RISK ASSESSMENT 
SOFTWARE TEST 


True Positive 


20 


21 


False Positive 


129 


57 


True Negative 


61 1 


683 


False Negative 


3 


2 


Sensitivity 


87.0% 


91.3% 


Specificity 


82.6% 


92.3% 


Pos PV 


13.4% 


26.9% 


Neg PV 


99.5% 


99.7% 


Odds Ratio 


31.6 


125.6 


Relative Risk 


27.4 


89.7 



Table 2. Performance comparison of Qualitative fFN ELISA Test and the Preterm 
Delivery Risk Assessment Software Test relative to risk of preterm delivery within 7 
days. 

Table 3 compares the performance of the two test relative to risk of preterm 
30 delivery within 14 days. Once again, the software decreases false positive test results 
when compared to the ELISA test, from 1 24 to 55, or about 53%. This decrease in 
false positive results is matched by the corresponding increase in true negative results, 
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10 



15 



20 



25 



30 



from 609 to 678. The number of true positive and false negative results were 
unchanged. Whilst the sensitivity of the test was unaffected, the specificity of the test 
rose nearly 10 points, increasing from 83.1% to 92.5%. As seen before, the positive 
predictive value nearly doubled, increasing from 16.8% to 31.3%, and the odds ratio 
and relative risk increased substantially from 24.6 to 61 .6 and from 20.7 to 44.7, 
respectively. 



MEASURE 


QUAL fFN ELISA TEST 


RISK ASSESSMENT 
SOFTWARE TEST 


True Positive 


25 


25 


False Positive 


124 


55 


True Negative 


609 


678 


False Negative 


5 


5 


Sensitivity 


83.3% 


83.3% 


Specificity 


83.1% 


92.5% 


Pos PV 


16.8% 


31 .3% 


Neg PV 


99.2% 


99.3% 


Odds Ratio 


24.6 


61 .6 


Relative Risk 


20.7 


44.7 



Table 3. Performance comparison of Qualitative fFN ELISA Test and the Preterm 
Delivery Risk Assessment Software Test relative to risk of preterm delivery within 14 
days. 

The following example is included for illustrative purposes only and are not 
intended to limit the scope of the invention. 

EXAMPLE 

Variable Selection and development of neural nets for predicting pregnancy related 
events and improvement of the performance of tests for fetal fibronectin 

The Fetal Fibronectin Enzyme Immunoassay (fFN ELISA) detects the presence or 

absence of fetal fibronectin (fFN) in cervicovaginal secretions (see, U.S. Patent No. 

5,468,619). Detection of fFN in cervicovaginal secretions of symptomatic pregnant 

women between 24 and 34 completed weeks gestation is associated with preterm 

delivery. This test is used to predict impending delivery within 7 or 14 days of 
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sampling. For prediction of delivery within 14 days for sampling of fFN, the negative 
result is greater than 99% accurate. The positive result is more difficult to interpret, 
and the positive predictive value is less than 20%. 

Neural networks were trained to assess the risk of preterm delivery using over 
700 examples of pregnant women who were symptomatic for preterm delivery. Each 
example contained a multitude of information about that patient, including symptoms, 
reproductive history and other factors. Neural networks were trained to recognize 
complex patterns of interactions between these factors that indicate when a woman is 
at risk for preterm delivery. These neural networks are contained in the Preterm 
Delivery Risk Assessment software, which augments the fFN ELISA test result by 
decreasing false positive observations. 
A. Variables 

The following are variables based on patient input data. Neural networks using 
all or selected subsets of these variables may be generated. Combinations of at least 
three of these variables may be used in conjunction with decision-support systems, 
particularly neural nets to predict risk of preterm delivery or impending delivery. The 
inputs for the variables are either yes, no, no answer, or a text input, such as age. 
The variables, listed by type are as follows: 

1 Age 

Ethnic origin variables: 

2 EthOrgl: Caucasian; 

3 EthOrg2: Black; 

4 Eth0rg3: Asian; 

5 EthOrg4: Hispanic; 

6 EthOrgS: Native American; and 

7 EthOrg6: Other than the above. 
Marital status variables: 

8 MarStl : Single; 

9 MarSt2: Married; 

10 MarSt3: Divorced/Separated; 
1 1 MarSt4: Widowed; 

12 MarStB: Living with partner; or 
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13 MarSt6: Other than those listed above. 
Education variables: 

14 EduO: Unknown; 

15 Edu1: <High School; 

5 16 Edu2: High School Graduate; or 

17 Edu3: College/trade. 
Patient complaint variables: 

18 PATIENT COMPLAINT #1 Patient has Uterine Contractions with or 

without pain; 

10 19 PATIENT COMPLAINT #2 Patient has intermittent lower abdominal 

pain, dull, low backache, pelvic pressure; 

20 PATIENT COMPLAINT #3 Patient has bleeding during the second or 
third trimester; 

21 PATIENT COMPLAINT #4 Patient has menstrual-like or intestinal 

15 cramping; 

22 PATIENT COMPLAINT #5 Patient has change in vaginal discharge or 
amount, color, or consistency; or 

23 PATIENT COMPLAINT #6 Patient is not "feeling right". 
Variables from physician tests and assessments: 

20 24 Pooling refers to visual assessment to determine whether amniotic 

fluid has leaked into the vagina (see, e.g. . Chapter 36, Section 18, p. 657 in Maternal 
Fetal Medicine: Principle and Practice , 2nd Edition, Creasy, R.F. et aL, eds., W.B. 
Saunders & Co. (1989)); 

25 Ferning refers to the results of a test to detect the pattern formed 

25 when amniotic fluid is present in a cervical sample smeared on a clean slide and allowed 
to air dry (see, e.g. . Chapter 36, Section 18, p. 657 in Maternal Fetal Medicine: 
Principle and Practice , 2nd Edition, Creasy, R.F. et aL, eds., W.B. Saunders & Co. 
(1989)); 

26 Nitrazine refers to results from a known test used to measure the pH 
30 of amniotic fluid that has leaked into the vagina (see, e.g. . Chapter 36, Section 18, p. 

657 in Maternal Fetal Medicine: Principle and Practice , 2nd Edition, Creasy, R.F. et al., 
eds., W.B. Saunders & Co. (1989)); 
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27 estimated gestational based (EGA) on last period (LMP); 

28 EGA by sonogram (SONO); 

29 EGA by Best-EGA is the best of the EGA by SONO and EGA by LMP 
determined as follows: 

5 if EGA by SONO is < 1 3 weeks, then EGA best is EGA 

SONO; 

if the difference by EGA by LMP and EGA by SONO is > 2 
weeks, then EGA best is EGA by SONO; otherwise EGA best is 
EGA by LMP; 

10 30 EGA at Sampling refers to the EGA when fFN sampled; 

31 CD INTERP, which refers to cervical dilatation (interpreted values - i.e. 
based on physicians estimates) where the number will be between 0 and 10 cm and is 
determined from the physicians response; 

32 Gravity, which refers to the number of time woman has been 

15 pregnant; 

33 Parity-term, which refers to the number of term births; 

34 Parity-preterm, which refers to the number of preterm births; 

35 Parity-abortions, which refers to the number of pregnancies ending in 
spontaneous or elective abortions; 

20 36 Parity-living, which refers to the number of living children; 

37 Sex within 24 hrs prior to sampling for fFN; 

38 Vaginal bleeding at time of sampling; 

39 Cervical consistency at time of sampling; and 

40 UC INTERP, which refers to uterine contractions per hour as 
25 interpreted by the physician. 

Complications 

41 0 COMP No previous pregnancies; 

42 1 COMP have had at least one previous pregnancy without 
complications; 

30 43 2nd comp at least one preterm delivery (delivery prior to 35 weeks); 

44 3rd comp at least one previous pregnancy with a premature rupture of 
membrane (PROM); 
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45 4th comp at least one previous delivery with incompetent cervix; 

46 5 COMP at least on previous pregnancy with pregnancy induced 
hypertension (PIH)/preeclampsia; 

47 6 COMP at least one previous pregnancy with spontaneous abortion 
5 prior to 20 weeks; 

48 OTHER COMP at least one previous pregnancy with a complication 
not listed above; and 

49 RESULT - fFN ELISA qualitative test result (if positive value is 1, if 
negative value is 0). 

10 The variable selection protocol has been applied to these variables for selected 

outcomes, and the results are set forth below. Exemplary neural nets are provided. 

B. A first set of neural networks demonstrating that the methods herein can be 
used to predict pregnancy related events 

EGA1-EGA4 

15 For these nets the preterm delivery defined as less than or equal to 34 weeks, 0 

days. The other nets herein (described below) define preterm delivery as less than or 

equal 34 weeks, 6 days. 

Data was collected from the over 700 test patients involved in a clinical trial of 

the assay described in U.S. Patent No. 5,468,619. Variable selection was performed 
20 without fetal fibronectin (fFN) test data. The final networks, designate EGA1-EGA4 

were trained with the variables set forth in the table below. 

EGA1 - EGA4 represent neural networks used for variable selection. For EGA1, 

the variable selection protocol was performed a network architecture with 8 inputs in 

the input layer, three processing elements in the hidden layer, and one output in the 
25 output layer. EGA2 is the same as EGA1, except that it is 9 inputs in the input layer. 

EGA3 has 7 inputs in the input layer, three processing elements in the hidden 

layer, and one output in the output layer. EGA4 is the same as EGA1 , except that it is 

8 inputs in the input layer. 

The variables selected are as follows: 
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EGA1 


EGA2 


L_ EGA3 


EGA4 




fFN 




fFN 


Ethnic Origin 1 
(Caucasian) 


Ethnic Origin 4 
(Hispanic) 


EGA Sonogram 


Marital Status 5 

/Itv/inn ia/i+Ki narfnorl 

Uiving wixn panncr ^ 


EGA Best 
(physician's determination of 
estimated gestational age) 


Marital Status 6 
(other) 


EGA Sampling j 


EGA Best 


Cervical dilation interpretation 


Cervical dilation interpretation 


Vaginal bleeding 
(at time of sampling) 


Vaginal bleeding 
(at time of sampling) 


1 complications 
(prev. preg w/o complications) 


Other complications 
(prev. preg. w complications) 


Other complications 
(prev. preg. w complications) 





EGA = estimated gestational age 



Final consensus network performance 



20 



25 



Net 


TP 


TN 


FP 


FN 


SN 


SP 


PPV 


NVP 


OR 


EGA1 


35 


619 


92 


17 


67.3 


87.0 


27.6 


97.3 


6.0 


EGA2 


37 


640 


71 


15 


71.2 


90.0 


34.3 


97.7 


7.9 


EGA3 


36 


602 


109 


16 


69.2 


84.7 


24.8 


97.4 


5.1 


EGA4 


32 


654 


57 


20 


61.5 


92.0 


36.0 


87.0 


8.9 


fFN 


31 


592 


119 


21 


59.6 


83.3 


20.7 


96.6 


7.3 



EG A = estimated gestational age (less than 34 weeks); TP = true positives; TN = true 
negatives; FP = false positives; FN = false negative; SN = sensitivity; SP = specificity;, 
PPV = positive predictive value; NPV = negative predictive value; OR = odds ratio (total 
30 number correct/total number correct answers); and fFN = the results from the ELISA 
assay for fFN. 
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The results show that the network EGA4, the neural net that includes seven 
patient variables and includes the fFN ELISA assay and that predicts delivery at less 
than 34 weeks, has far fewer false positives than the fFN ELISA assay. In addition, the 
number of false positives was reduced by 50%. Thus, incorporation of the fFN test into 
5 a neural net improved the performance of the fFN ELISA assay. All of the neural nets 
performed better than the fFN test alone. 

Thus, the methods herein, can be used to develop neural nets, as well as other 

decision-support systems, that can be used to predict pregnancy related events. 

C. Neural network prediction of delivery before 35 completed weeks of 
10 gestation -EGA5 and EGA6 

The fFN-NET database was used for all the experiments; organization of 

variables and order of variables was the same as described herein. Two variable 

selection runs were performed on the training data to determine the important variables 

to be used for the consensus runs. In each of the runs the hidden layer contained 5 

15 processing elements. This choice was based on the use of the variable selection 

process to determine the best size of the hidden layer. Variable selection was run with 
different numbers of hidden units in the neural network. The performance of the final 
selection of variables was compared for each different hidden layer configuration. Five 
hidden units were found to give the best performance. Each run used a partition of 5 

20 and a consensus of 10 networks. The top 10 variables were 

examined during the run before a variable was selected to be in the selection. 
During these runs the biochemical test variable, fFN result, was not included 
in the possible variables for variable selection. 

The resulting choices of variables were then re-evaluated using a consensus 

25 of 20 networks so that the two separate runs could be compared on an equal 
basis. Then the fFN result variable was added to the selected variables and 
the selections were re-evaluated using a consensus of 20 networks. This 
allowed the effect of the biochemical test on the performance to be 
determined. The final consensus training runs, using 8 networks, were made using all 

30 available data for training and the best performing set of variables from the above 
evaluations with the fFN result variable included. 
1. Variable selection 
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Using the same database described above for EGA1-EGA4, the variable selection 
protocol was applied as described above, except that the variable selection procedure 
was applied in the absence of the fFN test result. Since it is known that the results of 
this test are highly predictive of preterm or impending delivery, it was excluded from 
5 the variable selection procedure in order to eliminate its overriding influence, and to 
thereby select the important variables from among the other 48 variables. 

Application of the variable selection procedure to the 48 variables resulted in 
selection of the following variables: 





1. 


Ethnic Origin 1: Caucasian (i.e., yes or no); 


10 


2. 


Marital Status 5: living with partner (yes or no); 




3. 


EGA by sonogram 




4. 


EGA at sampling 




5. 


estimated date of delivery by best 




6. 


cervical dilatation (CM) 


15 


7. 


Parity-preterm 




8. 


vaginal bleeding at time of sampling 




9. 


cervical consistency at time of sampling ; and 




10. 


previous pregnancy without complication. 



2. Neural nets 

20 Using these variables two consensus networks were trained. One, designated 

EGA5, was trained without including the results of the fFN ELISA test result, and the 
other, designated EGA6, was trained with the results of the fFN ELISA test result. 

Fig. 12, which represents EGA6, is a schematic diagram of an embodiment of 
one type of neural network 10 trained on clinical data of the form used for the 

25 consensus network (Fig. 14) of a plurality of neural networks. The structure is stored 
in digital form, along with weight values and data to be processed in a digital computer. 
This neural network 10 contains three layers, an input layer 12, a hidden layer 14 and 
an output layer 16. The input layer 12 has eleven input preprocessors 17-27, each of 
which is provided with a normalizer (not shown in the figure, see table below), which 

30 generates a mean and standard deviation value to weight the clinical factors and which 
are input into the input layer. The mean and standard deviation values are unique to 
the network training data. The input layer preprocessors 17-27 are each coupled to 
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first and second and third processing elements 28, 29 and 30, of the hidden layer 14 
via paths 31-41, 42-52 and 53-63 so that each hidden layer processing element 28, 29 
and 30 receives a value or signal from each input preprocessor 17-27. Each path is 
provided with a unique weight based on the results of training on training data. The 
5 unique weights 64-74, 75-85, and 86-96 (see, also Table below) are non-iinearly 

related to the output and are unique for each network structure and initial values of the 
training data. The final value of the weights are based on the initialized values assigned 
for network training. The combination of the weights that result from training 
constitute a functional apparatus whose description as expressed in weights produces a 
10 desired solution, or more specifically a risk assessment for preterm delivery before 35 
weeks. 

The hidden layer 14 is biased by bias weights 97, 98 and 99 provided via paths 
100, 101, and 102 to the processing elements 28, 29 and 30. The output layer 16 
contains two output processing elements 103, 104. The output layer 16 receives input 

15 from the hidden layer processing elements 28, 29 and 30 via paths 105-1 10. The 
output layer processing elements 103, 104 are weighted by weights 111-116. The 
output layer 1 6 is biased by bias weights 117, 118 provided via paths 1 1 9 and 1 20 to 
the processing elements 103 and 104. 

The preliminary risk of delivery before 35 completed weeks of gestation is the 

20 output pair of values A and B from the two processing elements 103 and 104. The 

values are always positive between zero and one. One of the indicators is indicative of 
a risk of preterm delivery. The other is an indicator of the absence of such risk. While 
the output pair A, B provide generally valid indication of risk, a consensus network of 
trained neural networks provides a higher confidence index. EGA 6 contains 8 such 

25 trained neural networks. 

The following tables set forth the values of the individual weights for each of the 



8 consensus networks, designed EGA6_0 through EGA6_7. 
EGA 6 0 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


1st 


2nd 
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0 


0.412437 


-0.143143 


-1 .885393 


-0.9598620 


0.945025 


1 


2.041 149 


-0.021533 


0.162966 


-4.839373 


4.875033 


2 


1.224530 


0.971002 


-0.590964 


-2.524601 


2.524054 


3 


0.575975 


-3.249891 


-2.814656 


2 583483 


-2 561 113 

m* * w w III w 


4 


0.784864 


0.600535 


-0.300794 






5 


1.075542 


0.1601 136 


0.549237 






6 


-1.047227 


0.047396 


0.905172 






7 


-0.966051 


0.163156 


0.630888 








-0.193761 


-0.149381 


0.163185 






9 


-0.680552 


-2.362585 


1.365873 






10 


1.010706 


-3.633732 


-1.443890 






11 


1.728520 


-0.590057 


0.878588 







EGA 6 1 



Input layer 


hid 


den layer (nodes) 


output layer (nodes) 


node/weight 


1st 


2nd 


3rd 


1st 


2nd 


0 


2.675421 


-0.552641 


0.673642 


0.183663 


0.197713 


1 


-1.181347 


0.284937 


0.720041 


-3.170281 


3.095180 


2 


-0.178288 


-1.102137 


0.655263 


3.795940 


-3.747696 


3 


1 .048956 


-0.941387 


-1.733601 


-6.612447 


6.498429 


4 


0.033454 


0.927974 


2.987905 






5 


-1.161823 


1.217736 


1.014796 






6 


6.168329 


2.549298 


-1.321217 






7 ! 


-1.560728 


-1.637513 


-1.160331 






8 


1.671384 


3.395848 


-0.117778 






9 


0.416004 


1.452099 


-0.246268 






10 


-2.228914 


1.834281 


0.748248 








-3.687070 


1.693113 


-0.492244 
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EGA 6 2 



Input layer 


hidden layer (nodes) 


output layer (nodes) 


node/weight 


1st 


2nd 


3rd 


1st 


2nd 


0 


-1.013347 


1.392476 


3.390216 


1.093532 


-1,084186 


1 


-3.020375 


0.554074 


2.172394 


-1.633913 


1.632363 


2 


-0.899928 


1.928149 


0.466793 


-3.099829 


3.091530 


3 


-8.108200 


0.583508 


0.030467 


-2.860816 


2.845121 


4 


3.260629 


9.249855 


0.577971 






5 


-0.567385 


1.008019 


0.196682 






6 


-2.382355 


-2.942121 


0.568323 






7 


-1.996632 


-2.203792 


-0.852693 






B 


0.217054 


-0.230021 


-0.710703 






9 


0.380832 


-0.276078 


-1.551226 






10 


1.933148 


0.603005 


-0.856832 






11 


-1.922944 


-1.396864 


-2.356188 







EGA 6_3 



Input layer 


hidden layer (nodes) 


output layer (nodes) 


node/weight 


1st 


2nd 


3rd 


1st 


2nd 


0 


1.493395 


-2.294246 


2.173191 


-1.417536 


1.413825 


1 


3.959154 


0.635345 


0.976585 


-2.381441 


2.355649 j 


2 


0.396474 


-1.310699 


0.708136 


2.652994 


-2.638396 


3 


-0.404996 


-0.906109 


1.164319 


-3.176520 


3.136459 


4 


-0.113969 


-0.611 193 


-0.896189 






5 


0.665321 


-1.422789 


0.184973 






6 


1 .628547 


2.765793 


0.315556 






7 


-0.673276 


1.645794 


-0.975604 






8 


-2.422190 


1.272992 


0.612878 






9 


-1.494859 


2.990876 


0.002188 
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Input layer 


hidden layer (nodes) 


output layer (nodes) 


node/weight 


1st 


2nd 


3rd 


1st 


2nd 


10 


-0.316486 


-0.614556 


-0.993159 






11 


-3.208810 


-0.869353 


-3.219709 







EGA 6 4 



Input layer 


hid 


den layer (nod 


es) 


output la\ 


fer (nodes) 


node/weight 


1st 


2nd 


3rd 


1st 


2nd 


0 


1.595199 


-1.400935 


-1.254950 


-1.033706 


1.017989 


1 


1.597543 


1.434936 


-1.886380 


-3.899452 


3.915186 | 


2 


0.424391 


-0.524230 


0.974168 


2.759211 


-2.750812 


3 


1.340851 


0.063071 


-5.226755 


-2.077351 


2.087066 s 


4 


0.145379 


-3.090206 


-1.188423 






5 


0.569193 


-1.5561 14 


-1.835809 






6 


0.380544 


3.770102 


-1.193652 






7 


-0.41461 1 


2.391878 


-0.326348 






8 


0.082901 


0.821397 


-2.173482 






9 


-0.893175 


0.099641 


-1.615205 






10 


0.312568 


-0.034908 


-1 .900884 






11 


-1.068789 


1 .023022 


-1.393905 







EGA65 



Input layer 


hidden layer (nodes) 


output layer (nodes) 


node/weight 


1st 


2nd 


3rd 


1st 


2nd 


0 


2.503198 


-2.428604 


-0.130730 


-2.186942 


2.173897 


1 


-2.192063 


-3.125744 


3.638620 


-2.776665 


2.660086 


2 


1.579702 


0.833396 


1.472541 


2.737514 


-2.713886 


3 


-0.067358 


0.422544 


-1.196156 


-1.586596 


1.647172 


4 


1.298254 


-3.568407 


-1.013145 






5 


1.992165 


-3.716873 


-0.868908 
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Input layer 


hidden layer (nodes) 


output layer (nodes) 


node/weight 


1st 


2nd 


3rd 


1st 


2nd 


6 


-4.089348 


2.595805 


3.020147 






7 


-2.734360 


2.001578 


-0.018092 






8 


-1.668519 


-0.383332 


-3.587072 






9 


-1.886910 


0.268403 


-0.229832 






10 


-1.519840 


-1.147216 


1.671855 






1 1 


-1.200146 


3.289453 


-4.163397 







EGA 6 6 



Input layer 


hid 


den layer (nodes) 


output lav 


'er (nodes) 


node/weight 


1st 


2nd 


3rd 


1st 


2nd 


0 


-1.443015 


0.865813 


0.382970 


-2.388151 


2.408045 


1 


-1.582839 


0.593947 


0.830775 


4.015757 


-4.056962 


2 


-1.119793 


-0.355416 


0.803208 


-2.574057 


2.594654 


3 


2.549989 


0.295836 


0.454763 


-3.381956 


3.430132 


4 


-3.080358 


-3.033361 


1.023391 






5 


-2.302934 


0.508087 


-0,703378 






6 


-0.040867 


-2.352165 


-1.982702 






7 


1.082370 


3.718414 


-4.853944 






8 


-0.564883 


-4.419714 


-2.375676 






9 


0.953993 


-2.047337 


-0.481060 






10 


-1.062311 


0.216755 


-2.037935 






11 


1.488106 


-3.616466 


-0.630520 







EGA 6 7 



Input layer 


hidden layer (nodes) 


output lav 


f er (nodes) 


node/weight 


1st 


2nd 


3rd 


1st 


2nd 


0 


1.622433 


1.633779 


-3.852473 


-0.748768 


0.742163 


1 


0.043906 


-0.351661 


-2.431170 


-3.003003 


2.983215 
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input layer 


hidden layer (nod 


es) 


output layer (nodes) j 


node/weiaht 


1 St 


2nd 


3rd 


1 St 


2nd 


2 


0.73221 3 


-O.ool 362 


-0.746753 


-2.21 8790 


2.1 84970 


3 


-2 027060 


1 301339 


-1 768983 


O ,U3^30 1 




4 


1.521622 


1.790975 


-0.154270 






5 


1.677837 


-0.625462 


0.730582 






6 


-1.347791 


-4.165056 


-0.685942 






7 


-1.774773 


5.494371 


1 .034300 






8 


-0.827799 


1.789396 


0.538103 






9 


-0.509971 


-0.183482 


1.543398 






10 


0.605369 


2.345229 


1.277570 






11 


0.691960 


-3.950886 


2.871648 







The EGA6 preprocessing information is the same for each of the 8 neural 
networks in the consensus. The input is preprocessed by subtracting the mean 
value and dividing by the standard deviation. 





Node 


Mean 


Standard Deviation 


5 


1 


0.399738 


0.490166 




2 


0.01 1796 


0.108036 




3 


30.593335 


4.979660 




4 


30.709605 


5.405745 




5 


30.278038 


2.976036 


0 


6 


0.490092 


0.667659 




7 


0.178244 


0.471996 




8 


0.198946 


0.508406 




9 


1.823197 


0.757205 




10 


0.399738 


0.490166 


5 


11 


0.195282 


0.396677 
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EGA5 is a set of 8 consensus networks trained similarly to EGA6, except 

that the input variables did not include the variable representing the result of the 

fFN ELISA test. This network can be used as a point of care application to give 

immediate result to the clinician rather than the 24 to 48 hours required to 

5 process the fFN sample. 

D. Neural network prediction of risk of delivery within 7 days- EGAD7 
and EGAD7F 

1 . Variable selection 

Using the same database described above for EGA1-EGA6, the variable 
10 selection protocol was applied to prediction of the risk for delivery within 7 days 

of sampling for the fFN test. As noted above for EGA5 and EGA6, the variable 

selection procedure was applied in the absence of the fFN test result. 

Application of the variable selection procedure to the 48 variables resulted in 

selection of the following variables: 
15 1. Ethnic Origin 1: Caucasian ( i.e. , yes or no); 

2. Uterine contractions with or without pain ( i.e. , yes or no); 

3. Parity-abortions; 

4. Vaginal bleeding at time of sampling; 

5. Uterine contractions per hour; 
20 6. No previous pregnancies. 

2. Neural nets 
Using these variables two consensus networks were trained. One, 
designated EGAD7 was trained without including the results of the fFN ELISA 
test result, and the other, designated EGAD7f, was trained with the results of 
25 the fFN ELISA test result. 

Fig. 13, which represents EGA7f, is a schematic diagram of an 
embodiment of the neural network 10 trained on clinical data of the form used 
for the consensus network (Fig. 14) of a plurality of neural networks. The 
structure is stored in digital form, along with weight values and data to be 
30 processed in a digital computer. This neural network 10 contains three layers, 
an input layer 12, a hidden layer 14 and an output layer 16. The input layer 12 
has seven input preprocessors 17-23, each of which is provided with a 
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normalizer {not shown in the figure, see table below) which generates a mean 
and standard deviation value to weight the clinical factors which are input into 
the input layer. The mean and standard deviation values are unique to the 
network training data. The input layer preprocessors 17-23 are each coupled to 
5 first, second, third, fourth and fifth processing elements 24-28, respectively, of 
the hidden layer 14 via paths 29-35, 36-42, 43-49, 50-56, and 57-63 so that 
each hidden layer processing element 24-28, receives a value or signal from 
each input preprocessor 17-23. Each path is provided with a unique weight 
based on the results of training on training data. The unique weights 64-70, 71- 

10 77, 78-84, 85-91 and 92-98 (see, also Table below) are non-linearly related to 
the output and are unique for each network structure and initial values of the 
training data. The final value of the weights are based on the initialized values 
assigned for network training. The combination of the weights that result from 
training constitute a functional apparatus whose description as expressed in 

15 weights produces a desired solution, or more specifically a risk assessment of 
delivery within 7 days of sampling for the fFN ELISA test. 

The hidden layer 14 is biased by bias weights 99, 100, 101, 102 and 
103 provided via paths 104, 105, 106, 107 and 108 to the processing elements 
24, 25, 26, 27 and 28. The output layer 16 contains two output processing 

20 elements 109, 110. The output layer 16 receives input from the hidden layer 
processing elements 24-28 via paths 111-120. The output layer processing 
elements 109, 1 10 are weighted by weights 121-130. The output layer 16 is 
biased by bias weights 131, 132 provided via paths 133 and 134 to the 
processing elements 109 and 110. 

25 The preliminary risk of delivery within 7 days from sampling for the fFN 

ELISA test is the output pair of values A and B from the two processing 
elements 109 and 110. The values are always positive between zero and one. 
One of the indicators is indicative of a risk of delivery within 7 days. The other 
is an indicator of the absence of such risk. While the output pair A, B provide 

30 generally valid indication of risk, a consensus network of trained neural networks 
provides a higher confidence index. EGAD7f contains 8 such trained neural 
networks. 
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The following tables set forth the values of the individual weights for 

each of the 8 consensus networks, designated EGAD7fO through EGAD7f7: 
EGAD7fO 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


-0.204716 


1.533574 


1.452831 


0.129981 


-1.784807 


0.854229 


-0.883808 


1 


-1.843673 


1.957059 


-2.668371 


-0.551016 


1.505628 


-5.294533 


5.303048 


2 


-1.324609 


0.258418 


-1.280479 


-0.476101 


0.827188 


-7.468771 


7.514580 


3 


-1.281561 


1.697443 


6.865219 


4.212538 


-1.953753 


-5.082050 


5.003566 


4 


-1.159086 


-0.345244 


-4.689749 


-0.406485 


1.027280 


4.014138 


-4.006929 


5 


-2.042978 


0.182091 


2.612433 


2.399196 


-1.397453 


-4.105859 


4.105161 


6 


-4.076656 


1.416529 


0.979842 


-2.589272 


0.068466 






7 


-0.499705 


-1.383732 


-2.4 11 544 


0.173131 


-1.919889 







EGAD7f 1 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


1.522090 


6.396365 


1.750606 


0.650769 


0.673423 


0.282480 


-0.222861 


1 


1.930314 


0.027271 


0.386927 


1.602559 


3.495371 


-5.126995 


4.888618 


2 


1.578675 


-0.445222 


0.352425 


1.305894 


1.703156 


-3.751147 


3.752025 


3 


1.B21893 


6.258497 


1.140159 


1.363783 


-0.717021 


-5.496184 


5.687717 


4 


-4.599618 


0.218248 


0.385593 


0.945824 


0.644622 


7.713794 


-8.054935 


5 


-2.755846 


-1.799000 


2.162089 


1 .730335 


-0.388646 


-3.429169 


3.706028 


6 


0.524701 


1 .669467 


1.741620 


3.956515 


4.717868 






7 


-2.089663 


-0.190423 


-1.736970 


0.085315 


-1.010295 
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EGAD7f2 



Input 
layer 


hidden layer (nodes! 


output layer (nodes) 


node 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


0.554749 


4.029042 


1.041783. 


0.687361 


2.078268 


0.718456 


-0.756554 


1 


0.3143S5 


-1.614025 


4.560114 


-0.197290 


2.352322 


3.339842 


-3.185465 


2 


-1.992577 


-1.810437 


2.067243 


-0.021868 


0.041441 


-5.596330 


5.470991 


3 


-4.762585 


-6.021220 


3.627642 


3.505088 


1.221308 


0.815486 


-0.906961 


4 


8.422636 


-1.088322 


-1.229308 


-2.513499 


0.344056 


-4.076351 


4.165072 


5 


-0.547021 


-6.256763 


1.108255 


1.341978 


-0.074222 


-7.385492 


7.372295 


6 


0.581056 


-2.916328 


0.639607 


0.894802 


2.365492 






7 


1.260577 


-1.583044 


0.882731 


-1.113407 


-1.657523 







EGAD7f3 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/we 
ight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


1.258939 


0.778115 


1.117508 


-5.828234 


3.275221 


-0.174440 


0.260818 


1 


1.038074 


0.395096 


-1.080656 


-0.580291 


-1.077984 


-6.546609 


6.515666 


2 


-2.174144 


0.453939 


-0.677622 


-1.330231 


-0.383479 


-8.061748 


8.067432 


3 


0.608410 


2.262108 


9.263388 


4.024162 


0.949009 


4.938700 


-5.060233 


4 


1.443697 


-1.530076 


-0.812837 


1 .549062 


-1.594324 


5.420476 


-5.517191 


5 


-1.437676 


0.749049 


5.493512 


-2.797146 


-2.056666 


-5.085781 


5.127757 


6 


0.778191 


1.397835 


-3.635368 


2.191902 


-2.403500 






7 


-1.776540 


-0.675587 


0.1 15710 


0.388203 


-1.363938 







EGAD7f4 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


-1.839879 


0.255905 


3.002103 


0.886848 


-0.485949 


-1.461668 


1.340040 


1 


-1.336228 


-3.428058 


0.666937 


-1.072766 


-0.372897 


-1.862627 


1.815599 


2 


0.062547 


0.489211 


0.946443 


-3.642373 


3.973801 


5.835287 


-5.699656 


3 


1 .888678 


1.928167 


4.900952 


1.928106 


-1.866227 


-5.463729 


5.463984 
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Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


4 


-5.217631 


-1.441 138 


-4.1 14171 


0.629958 


-1.615146 


-5.726771 


5.763464 


5 


♦0.631546 


1 .735842 


1.158419 


0.638580 


-3.276926 


•7.193156 


7.177080 


6 


-3.109977 


-0.377960 


1.372646 


2.625961 


-1.700064 






7 


-0.070132 


1.763962 


-2.234798 


-1.165563 


-1.845262 







EGAD7f5 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


-1.456277 


1.321048 


1.214385 


0.069355 


-0.206125 


-1 .581 118 


1.811097 


1 


1.988970 


-2.788917 


1.700144 


-3.790842 


0.760984 


-3.282460 


2.842431 


2 


-0.889522 


-1 .748239 


0.798888 


-0.481237 


0.24B333 


-6.391959 


6.435954 


3 


15.258006 


0.809204 


4.07181 1 


-3.751193 


-6.873492 


-6.817300 


6.829902 


4 


-18.202002 


-2.000871 


0.021785 


0.812317 


0.713510 


6.157183 


-6.412641 


5 


0.440615 


-0.470067 


-1.578267 


-0.216803 


-3.315356 


-7.015062 


6.902B92 


6 


-1.931575 


0.510900 


1.162408 


-2.528233 


1.405955 






7 


3.758462 


-0.570789 


-6.338710 


0.877703 


-0.985724 







EGAD7f6 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/w 
eight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


1.512437 


-0.333348 


-0.557454 


-0.790704 


0.049061 


-0.918761 


0.804829 


1 


-0.704182 


-0.032274 


-3.201322 


-0.966885 


-0.213225 


-2.987857 


2.999401 


2 


0.443652 


-0.736894 


-0.713164 


-0.709163 


-0.725865 


-5.682138 


5.676150 


3 


2.734173 


0.555570 


-2.071605 


7.636067 


-7.109310 


4.989255 


-4.851893 


4 


-4.066469 


-0.039688 


0.313027 


-0.265136 


0.152398 


-4.107172 


4.101486 


5 


0.943337 


-0.658673 


-0.079748 


3.091015 


-5.459067 


-5.247225 


5.231175 


6 


-0.211375 


0.247671 


-2.400778 


2.663087 


-1.717437 






7 


-1.291067 


-4.507938 


1.526173 


-0.139780 


-0.451653 
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EGAD7f7 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5 th 


1st 


2nd 


0 


0.580523 


0.319374 


-0.660897 


1.072931 


-0.522045 


-0.833235 


1.016355 


1 


0.432923 


3.916608 


0.386343 


-1.324510 


-1.566712 


-4,472839 


4.433871 


2 


-0.312324 


3.099275 


0.344633 


-3.254393 


-1.081 114 


-4.873536 


4.919722 


3 


4.019378 


-5.440501 


-9.105190 


1.955846 


-2.152612 


4.971 172 


-5.215318 


4 


-0.355344 


0.495595 


0.543102 


-2.001959 


-0.989721 


-3.436097 


3.478752 


5 


-1.585942 


-3.885213 


-2.778485 


1.068593 


-1.697807 


-4.098137 


4.165162 


6 


-0.209687 


-0.646458 


-2.399903 


0.177487 


2.339257 






7 


■8.951553 


-1.471208 


0.725651 


-2.732204 


1.538870 







The EGAD7F preprocessing information is the same for each of the 
8 neural networks in the consensus. The input in preprocessed by 
subtracting the mean value and dividing by the standard deviation. 



Node 


Mean 


Standard Deviation 


1 


0.399738 


0.490166 


2 


0.517693 


0.500015 


3 


0.621232 


1 .030720 


4 


0.198946 


0.508406 


5 


2.144928 


2.291734 


6 


0.281782 


0.450163 


7 


0.195282 


0.396677 



EGAD7 is a set of 8 consensus networks trained similarly to 
5 EGAD7f , except that the input variables did not include the variable 

representing the result of the fFN ELISA test. This network can be used 
as a point of care application to give immediate result to the clinician 
rather than the 24 to 48 hours required to process the fFN sample. 
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E. Neural network prediction of risk of delivery within 14 days- 
EGAD14f and EGAD 14 

1 . Variable selection 

Using the same database described above for EGA1-EGAD7, the variable 

5 selection protocol was applied to prediction of the risk for delivery within 14 

days of sampling for the fFN test. As noted above for EGA5, EGA6 and 

EGAD7, the variable selection procedure was applied in the absence of the fFN 

test result. Application of the variable selection procedure to the 48 variables 

resulted in selection of the following variables: 

10 1. Ethnic Origin 4: Hispanic ( i.e. , yes or no); 

2. Marital Status 5: living with partner; 

3. Uterine contractions with or without pain ( i.e. , yes or no); 

4. Cervical dilatation; 

5. Uterine contractions per hour; 
15 6. No previous pregnancies. 

2. Neural nets 
Using these variables two consensus networks were trained. One, 
designated EGAD 14 was trained without including the results of the fFN ELISA 
test result, and the other, designated EGAD14f, was trained with the results of 

20 the fFN ELISA test result. 

Fig. 13, which represents EGAD14f (as well as EGAD7f), is a schematic 
diagram of an embodiment of the neural network 10 trained on clinical data of 
the form used for the consensus network (Fig. 14) of a plurality of neural 
networks. The structure is stored in digital form, along with weight values and 

25 data to be processed in a digital computer. This neural network 10 contains 
three layers, an input layer 12, a hidden layer 14 and an output layer 16. The 
input layer 12 has seven input preprocessors 17-23, each of which is provided 
with a normalizer (not shown in the figure, see Table, below) which generates a 
mean and standard deviation value to weight the clinical factors which are input 

30 into the input layer. The mean and standard deviation values are unique to the 
network training data. The input layer preprocessors 17-23 are each coupled to 
first, second, third, fourth and fifth processing elements 24-28, respectively, of 
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the hidden layer 14 via paths 29-35, 36-42, 43-49, 50-56, and 57-63 so that 
each hidden layer processing element 24-28, receives a value or signal from 
each input preprocessor 17-23. Each path is provided with a unique weight 
based on the results of training on training data. The unique weights 64-70, 71- 
5 77, 78-84, 85-91 and 92-98 (see, also Table below) are non-linearly related to 
the output and are unique for each network structure and initial values of the 
training data. The final value of the weights are based on the initialized values 
assigned for network training. The combination of the weights that result from 
training constitute a functional apparatus whose description as expressed in 

10 weights produces a desired solution, or more specifically a risk assessment of 
delivery within 14 days of sampling for the fFN ELISA test. 

The hidden layer 14 is biased by bias weights 99, 100, 101, 102 and 
103 provided via paths 104, 105, 106, 107 and 108 to the processing elements 
24, 25, 26, 27 and 28. The output layer 16 contains two output processing 

15 elements 109, 110. The output layer 16 receives input from the hidden layer 
processing elements 24-28 via paths 111-120. The output layer processing 
elements 109, 1 10 are weighted by weights 121-130. The output layer 16 is 
biased by bias weights 131, 132 provided via paths 133 and 134 to the 
processing elements 109 and 110. 

20 The preliminary risk of delivery within 14 days from sampling for the fFN 

ELISA test is the output pair of values A and B from the two processing 
elements 109 and 1 10. The values are always positive between zero and one. 
One of the indicators is indicative of a risk of delivery within 14 days. The other 
is an indicator of the absence of such risk. While the output pair A, B provide 

25 generally valid indication of risk, a consensus network of trained neural networks 
provides a higher confidence index. EGAD14f contains 8 such trained neural 
networks. 

The following tables set forth the values of the individual weights for 
each of the 8 consensus networks, designed EGAD14fO through EGAD14f7. 
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EGAD14fO 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/w 
eight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


-0.191126 


1.174059 


0.810632 


0.148573 


-2.437188 


0.106355 


-0.108766 


1 


-2.921661 


-0.713076 


1.312931 


10.427816 


1.824513 


-2.220130 


2.198498 


2 


-0.848702 


1.614504 


2.640692 


-0.445807 


1.218097 


-2.016395 


2.005455 


3 


-1.008667 


0.138305 


1.372127 


0.788516 


-3.114650 


-4.365818 


4.349520 


4 


-1.422990 


-1.517308 


-1.632533 


-3.146550 


0.256047 


2.291882 


•2.293527 


5 


-2.588523 


-0.733381 


0.992748 


1.482687 


1.197727 


-4.864353 


4.861522 


6 


-3.611756 


-2.669159 


3.364100 


-1.806442 


0.833890 






7 


-0.516151 


-2.104245 


-2.052761 


-0.615030 


-1.621589 







EGAD14M 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


0.396502 


2.426709 


0.752911 


1.549394 


-0.064008 


-0.285667 


0.714618 


1 


1.248711 


2.179334 


-0.016570 


-0.040113 


2.457661 


-3.745954 


3.884410 


2 


1.912210 


0.937177 


-1.742286 


-2.094312 


-1.165847 


-4.912591 


4.966647 


3 


-1.018760 


-1.087528 


-0.344108 


0.384237 


-1.077692 


-7.433263 


7.309962 


4 


1 .090578 


-2.229295 


-0.890326 


-1.334206 


0.822185 


2.080292 


-2.595363 


5 


1.399831 


-5.077936 


-0.600345 


4.128439 


-1.715393 


5.481619 


-5.611861 


6 


2.241531 


-4. 673233 


-0.209741 


2.954158 


-4.565109 






7 


0.077090 


-0.194145 


-4.391311 


3.260038 


-2.360049 







EGAD14f2 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


0.286926 


1.855804 


0.103985 


-2.590399 


2.265841 


1.540065 


-1.592696 


1 


1.928731 


0.410516 


-2.015740 


1.017801 


2.088775 


2.433105 


-2.545955 


2 


-0.666312 


-1.178337 


1 .227737 


-1.471309 


1.922938 


-4.736276 


4.903823 


3 


-2.716156 


-2.328632 


-0.566546 


0.854688 


-0.448565 


-2.220462 


2.268171 
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Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


4 


0.654814 


-0.197945 


-2.256156 


-0.410249 


-0.792705 


-4.049918 


4.142265 


5 


-2.004537 


-3.451720 


3.311102 


1.787226 


-0.682330 


-3.930044 


4.036821 


6 


-0.947058 


-1.898302 


-0.131517 


4.187262 


2.272720 






7 


0.485620 


-0.138471 


1.038285 


-1.245135 


-6.442445 







EGAD 1 4f 3 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


1.199346 


1.135219 


2.839737 


-4.673778 


2.903983 


-0.702760 


0.935822 


1 


-1.274101 


1.559637 


1.386395 


-0.042351 


-0.874145 


-3.244763 


3.144603 


2 


-0.353335 


0.325171 


-1.677620 


-0.793429 


0.788584 


-4.933673 


4.849451 


3 


-0.678281 


-2.157454 


-3.084480 


1.009661 


0.327746 


3.306738 


-3.432135 


4 


1.116566 


0.128203 


-2.188180 


2.315793 


-1.815446 


4.993960 


-5.098751 


5 


-1.277371 


-0.415757 


-0.080374 


-0.694424 


-1.022B31 


-4.266839 


4.064770 


6 


-4.836841 


3.738553 


-0.703345 


0.271620 


-0.626113 






7 


-0.953257 


-0.463343 


1.314770 


-0.196871 


-2.372877 







EGAD14f4 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/w 
eight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


-1.810913 


-0.014B85 


0.167362 


-2.605120 


-0.205378 


-0.681096 


0.709641 


1 


5.080080 


1.259709 


0.430446 


0.680130 


-3.098744 


-3.611765 


3.644697 


2 


-0.414857 


-0.328851 


-0.335724 


5.756228 


1.904646 


4.377642 


-4.419249 


3 


0.525909 


1.767786 


-0.375093 


1.041263 


-0.566611 


-6.720907 


6.647904 


4 


-7.166096 


-0.912267 


-1.948366 


-1.117219 


-1.237101 


-2.355787 


2.337121 


5 


-4.340267 


-0.345630 


-0.077869 


3.853568 


-2.550077 


-2.249878 


2.171079 i 


6 


-2.586306 


-3.315458 


0.378838 


5.812339 


-3.619375 






7 


0.213139 


-1.546969 


-10.991954 


-1.1B6517 


-0.602957 
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EGAD14f5 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/w 
eight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


-2.439228 


0.954525 


1.242215 


-27.696498 


0.322283 


-2.017057 


2.095211 


1 


1.998281 


1.928331 


0.638520 


-1.415280 


1.871968 


6.487561 


-6.308325 


2 


-0.869648 


-0.994059 


0.768856 


0.368344 


1.457719 


-4.867902 


4.744858 


3 


0.295868 


-0.257773 


1.422994 


0.033843 


-4.658167 


-2.392888 


2.192236 


4 


-1.800394 


-2.612705 


-1.668799 


51.649234 


-0.537556 


1.222661 


-1.270161 


5 


0.992302 


-0.938952 


1.104910 


3.731820 


1.651959 


-1.649461 


1.594009 


6 


-1.787379 


-1.045545 


2.711432 


0.288323 


-0.572490 






7 


-0.374909 


-0,877122 


-1.918442 


214.812434 


-1.773228 







EGAD14f6 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/we 
ight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


3.984308 


-0.300188 


6.132831 


1 .776838 


1.182643 


-0.141300 


-0.062816 


1 


-2.478863 


0.891740 


-0.185527 


-0.442487 


1 .045499 


-5.041497 


4.985260 


2 


0.389668 


0.650328 


-2B9.318971 


0.651142 


0.169117 


-7.230831 


7.280185 | 


3 


0.370846 


0.503667 


21.787679 


1.820010 


-0.802930 


2.464335 


-2.250474 


4 


-0.950033 


-0.054657 


0.942573 


-1.024688 


-1.842654 


2.637713 


-2.636534 


5 


3.200645 


0.464231 


0.728644 


1.784671 


-5.371345 


-3.675622 


3.704625 


6 


0.647747 


2.560388 


-0.798268 


3.237414 


-4.493387 






7 


-1.276096 


-1 .593493 


66.059880 


0.493228 


-0.126844 







EGAD14f7 



Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


0 


0.888004 


0.521346 


-0.513845 


0.767983 


-0.956920 


-1.088033 


1 .264836 


1 


0.191409 


1.634987 


-0.771837 


-2.402982 


-1.003714 


-4.407106 


4.589468 


2 


2.233326 


0.767802 


-10.205298 


0.362276 


0.797006 


-4.385751 


4.466996 


3 


-0.588252 


-5.586697 


0.233647 


0.586147 


1 .589040 


5.286617 


-6.662157 


4 


-1.544910 


-0.829764 


0.624734 


-5.119879 


-0.276545 


-0.907527 


0.809701 
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Input 
layer 


hidden layer (nodes) 


output layer (nodes) 


node/ 
weight 


1st 


2nd 


3rd 


4th 


5th 


1st 


2nd 


5 


-0.361805 


0.397313 


-1.973167 


-2.953926 


-0.614287 


-5.146765 


5.284392 


6 


-0.136039 


-1.488352 


-3.541771 


3.717852 


-1.091340 






7 


-8.058644 


-1.997797 


1.520159 


-0.638158 


1.013775 







10 The EGAD14F preprocessing information is the same for each of the 8 

neural networks in the consensus. The input is preprocessed by subtracting the 
mean value and dividing by the standard deviation. 



20 



Node 


Mean 


Standard Deviation 


1 


0.152031 


0.359287 


2 


0.91796 


0.108036 


3 


0.517693 


0.500015 


4 


0.490092 


0.667659 


5 


2.144928 


2.291734 


6 


0.281782 


0.450163 


7 


0.195282 


0.396677 



EGAD14 is a set of 8 consensus networks trained similarly to EGAD14f, 
except that the input variables did not include the variable representing the result 
25 of the fFN ELISA test. This network can be used as a point of care application 
to give immediate result to the clinician rather than the 24 to 48 hours required 
to process the fFN sample. 

Since modifications will be apparent to those of skill in this art, it is 
30 intended that this invention be limited only by the scope of the appended claims. 
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WHAT IS CLAIMED: 

1. A method of assessment of the risk of preterm delivery or the risk 
of delivery within a selected period of time, comprising: 

(a) selecting a set of important selected variables by: 

5 (i) providing a first set of n candidate variables and a 

second set of selected important variables, wherein the second set 
is initially empty; 

(ii) taking candidate variables one at a time and evaluating 
each by training a decision-support system on that variable 

10 combined with the current set of selected important variables; 

(iii) selecting the best of the candidate variables, wherein 
the best variable is any one that gives the highest performance of 
the decision-support system, and if the best candidate variable 
improves performance compared to the performance of the 

15 selected important variables, adding it to the selected important 

variable set, removing it from the candidate set and continuing 
evaluating at step (ii), wherein the best candidate variable does not 
improve performance, the process is completed; and 

(b) training a decision-support system using the selected final set of 
20 important selected variables to produce a test for assessment of the risk of 

preterm delivery or risk of delivery within a selected time period,, wherein 
assessment of delivery within a selected period of time refers either to prediction 
of delivery at a particular gestational ago, or the risk of delivery within a given 
time interval. 

25 2. The method of claim 1, wherein in step (i) the candidate variables 

are obtained from patients and include historical data and/or biochemical data. 

3. The method of claim 1, wherein the method of diagnosis assesses 
the likelihood of preterm delivery. 

4. A method of improving the effectiveness of a diagnostic 

30 biochemical test for preterm delivery or for the risk of delivery within a selected 
period of time, comprising: 

(a) selecting a set of important selected variables by: 
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(i) providing a first set of n candidate variables and a 
second set of selected important variables, wherein the second set 
is initially empty; 

(ii) taking candidate variables one at a time and evaluating 
5 each by training a decision-support system on that variable 

combined with the current set of selected important variables; 

(iii) selecting the best of the candidate variables, wherein 
the best variable is any one that gives the highest performance of 
the decision-support system, and if the best candidate variable 

10 improves performance compared to the performance of the 

selected important variables, adding it to the selected important 
variable set, removing it from the candidate set and continuing 
evaluating at step (ii), wherein the best candidate variable does not 
improve performance, the process is completed; and 
15 (b) training a decision-support system using the selected final set of 

important selected variables and the biochemical test data to produce a test that 
is more effective in assessing the risk or preterm delivery or delivery within a 
selected time period than the biochemical test alone. 

5. The method of claim 4, wherein the candidate variables are 
20 responses to queries selected from the group consisting of: 
Age; 

Ethnic origin Caucasian; 
Ethnic origin Black; 
ethnic origin Asian; 
25 ethnic origin Hispanic; 

ethnic origin Native American; 

ethnic origin other than the Native American, Hispanic, Asian, Black, or 
Caucasian; 

marital status single; 
30 marital status married; 

marital status divorced or separated; 
marital status widowed; 
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marital status living with partner; 

marital status other than married, divorced/separated, widowed, or living 
with partner; 

education unknown; 
5 education less than high school; 

education high school graduate; 
education college or trade school; 

patient has Uterine Contractions with or without pain; 
patient has intermittent lower abdominal pain, dull, low backache pelvic 
10 pressure; 

patient has bleeding during the second or third trimester; 
patient has menstrual-like or intestinal cramping; 

patient has change in vaginal discharge or amount, color, or consistency; 
patient is not "feeling right"; 
15 pooling; 

ferning; 
nitrazine; 

estimated gestational age (EGA) based on last menstrual period (LMP); 
EGA by sonogram (SONO); 
20 EGA by best, wherein EGA by best refers to the best of EGA by SONO 

and EGA by LMP determined as follows: 

if EGA by SONO is < 13 weeks, then EGA best is EGA SONO; 
if the difference by EGA by LMP and EGA by SONO is > 2 
weeks, then EGA best is EGA by SONO; otherwise EGA best is 
25 EGA by LMP; 

EGA at sampling; 
cervical dilatation (CO); 
gravity; 
parity-term; 
30 parity-preterm; 

parity-abortions, wherein the number of abortions include spontaneous 
and elective abortions; 
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parity-living; 

sex within 24 hrs prior to sampling for fFN; 
vaginal bleeding at time of sampling; 
cervical consistency at time of sampling; 
5 uterine contractions per hour as interpreted by the physician; 

no previous pregnancies; 

at least one previous pregnancy without complications; 
at least one preterm delivery; 

at least one previous pregnancy with a premature rupture of membrane 
10 (PROM); 

at least one previous delivery with incompetent cervix; 
at least on previous pregnancy with pregnancy induced hypertension 
(PIH)/preeclampsia; 

at least one previous pregnancy with spontaneous abortion prior to 20 
15 weeks; and 

at least one previous pregnancy with a complication not listed above. 
6. The method of claim 4, wherein the candidate variables are 
responses to queries selected from the group consisting of: 
Caucasian; 
20 living with partner; 

EGA by sonogram; 
EGA at sampling; 

estimated date of delivery by best; 
cervical dilatation (CM); 
25 Parity-preterm; 

vaginal bleeding at time of sampling; 
cervical consistency at time of sampling ; and 
previous pregnancy without complication. 
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7. The method of claim 4, wherein the candidate variables are 
responses to queries selected from the group consisting of: 

Caucasion; 

Uterine contractions with or without pain; 
5 Parity-abortions; 

Vaginal bleeding at time of sampling; 
Uterine contractions per hour; and 
No previous pregnancies. 

8. The method of claim 5 f wherein the biochemical test is a test that 
10 detects fetal fibronectin in cervico/vaginal samples; determines the level of a 

local inflammatory product protein in cervico/vaginal samples; or estriol or 
estretol in saliva. 

9. The method of claim 1, wherein the candidate variables include 
biochemical test data. 

15 10. A method for assessing the risk of delivery prior to completion of 

35 weeks of gestation, comprising assessing a subset of variables containing at 
least three and up to all of the responses to the following queries; 
Ethnic Origin Caucasian; 
Marital Status living with partner; 
20 EGA by sonogram; 

EGA at sampling; 

estimated date of delivery by best; 

cervical dilatation (CM); 

parity-preterm; 
25 vaginal bleeding at time of sampling; 

cervical consistency at time of sampling; and 

previous pregnancy without complication, 
using a decision-support system that has been trained to assesses the risk of 
delivery prior to 35 weeks of gestation. 
30 1 1 . The method of claim 10, wherein the decision support system is a 

neural network. 
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12. The method of claim 10, wherein the decision-support system has 
been trained using a set of variables that do not include biochemical test data. 

13. In a computer system, a method for assessing the risk of delivery 
prior to completion of 35 weeks of gestation comprising: 

5 (a) collecting observation values reflecting presence and absence 

of specified clinical data factors and storing the observed clinical data factors in 
storage means of the computer system, the specified clinical data factors 
comprising at least four up to all of the factors selected from the group 
consisting of: 

10 Ethnic Origin Caucasian, Marital Status living with partner, EGA by 

sonogram, EGA at sampling, estimated date of delivery by best, cervical 
dilatation (CM), parity-preterm, vaginal bleeding at time of sampling, cervical 
consistency at time of sampling, and previous pregnancy without complication; 

(b) applying the observation values from the memory means to a 

15 first decision-support system trained on samples of the specified factors; and 
thereupon 

<c) extracting from the first decision-support system an output 
value, wherein the output value is a quantitative objective aid to assess the risk 
of delivery prior to 35 weeks of gestation. 
20 14. The method of claim 13, wherein the decision-support system 

comprises a neural network. 

15. The method of claim 13, wherein at least five factors are selected. 

16. The method of claim 13, wherein the clinical factors further 
comprise the result of a test that detects fetal fibronectin in cervico/vaginal 

25 samples; the result of a test that determines the level of a local inflammatory 
product protein in cervico/vaginal samples; or the result of a test that assesses 
estriol or estretol in saliva. 

17. The method of claim 16, wherein the selected factors include the 
result of the test. 
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18. The method of claim 17, further comprising: 

b1) applying said observation values from said memory means to a 
plurality of the first decision-support system, wherein each one of the first 
decision-support systems is trained on the samples of the specified factors with 
5 different starting weights for each training; 

d) extracting from the first decision-support system, output value 
pairs for each one of said first neural networks; and 

d) forming a linear combination of said first ones of said output 
value pairs and forming a linear combination of said second ones of said output 
10 value pairs, to obtain a confidence index pair, said confidence index pair being 
said quantitative objective aid. 

19. The method of claim 18, wherein the first decision support system 
is a neural network that comprises a three-layer network containing an input 
layer, a hidden layer and an output layer, the input layer having eleven input 

15 nodes, first, second and third second hidden layer nodes, a hidden layer bias for 
each hidden layer node, first and second output layer nodes in the output layer, 
and an output layer bias for each output layer node. 

20. The method of claim 18, wherein the first decision support 
system is a neural network and each of the plurality of first trained neural 

20 networks comprises a three-layer network comprising an input layer, a hidden 
layer and an output layer. 

21 . The method of claim 13, further comprising: 

b1) applying said observation values from said memory means to a 
plurality of the first decision-support system, wherein each one of the first 
25 decision-support systems is trained on the samples of the specified factors with 
different starting weights for each training; 

d) extracting from the first decision-support system, output value 
pairs for each one of said first neural networks; and 

d) forming a linear combination of said first ones of said output 
30 value pairs and forming a linear combination of said second ones of said output 
value pairs, to obtain a confidence index pair, said confidence index pair being 
said quantitative objective aid. 
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22. The method of claim 21, wherein the first decision support system 
is a neural network that comprises a three-layer network containing an input 
layer, a hidden layer and an output layer, the input layer having eleven input 
nodes, first, second and third second hidden layer nodes, a hidden layer bias for 

5 each hidden layer node, first and second output layer nodes in the output layer, 
and an output layer bias for each output layer node. 

23. The method of claim 21, wherein the first decision support 
system is a neural network and each of the plurality of first trained neural 
networks comprises a three-layer network comprising an input layer, a hidden 

10 layer and an output layer. 

24. In a computer system, a method for assessing the risk of delivery 
prior to completion of 35 weeks of gestation, comprising the steps of: 

(a) collecting observation values reflecting presence and absence 
of specified factors and storing the observation factors in storage means of the 

15 computer system, the specified factors comprising: Ethnic Origin Caucasian, 
Marital Status living with partner, EGA by sonogram, EGA at sampling, 
estimated date of delivery by best, cervical dilatation (CM), parity-preterm, 
vaginal bleeding at time of sampling, cervical consistency at time of sampling, 
and previous pregnancy without complication; 

20 (b) obtaining results from the patient of a test that detects fetal 

fibronectin (fFN) mammalian body tissue and fluid samples, and/or of a test that 
determines the level of a local inflammatory product protein in cervico/vaginal 
samples; and/or of a test that assesses estriol or estretol in saliva; 

(c) applying the observation values and the fFN test results from 
25 the memory means to a second neural network trained on samples of the 

specified factors and the test results; and thereupon 

(d) extracting from the second trained neural network an output 
value pair, the output value pair being a preliminary indicator for the risk of 
delivery prior to 35 weeks of gestation. 

30 25. The method of claim 24, further including the steps of: 

(d) applying the observation values and the relevant biochemical 
test results from the memory means to a plurality of the second neural networks, 
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each one of the first neural networks being trained on the samples of the 
specified factors with starting weights for each training being randomly 
initialized; 

(d1) extracting from each one of the first trained neural networks, 
5 output value pairs for each one of the first neural networks; and 

(e) forming a linear combination of the first ones of the output 
value pairs and forming a linear combination of the second ones of the output 
value pairs, to obtain a confidence index pair, the confidence index pair being a 
final indicator for the risk of delivery prior to 35 weeks of gestation. 
10 26. The method of claim 24, wherein the first trained neural network 

comprises a three-layer network containing an input layer, a hidden layer and an 
output layer, the input layer having eleven input nodes, first, second and third 
hidden layer nodes, a hidden layer bias for each hidden layer node, first and 
second output layer nodes in, the output layer, and an output layer bias for each 
15 output layer node. 

27. A method for assessing the risk for delivery in 7 or fewer days, 
comprising assessing a subset of variables containing at least three up to all of 
the following variables: 

Ethnic Origin Caucasian; 
20 Uterine contractions with or without pain; 

Parity-abortions; 

vaginal bleeding at time of sampling; 
uterine contractions per hour; and 
No previous pregnancies, 
25 using a decision-support system that has been trained to assess the risk of 
delivery within seven days. 

28. The method of claim 27, wherein: 

the variables further include the result of a test for to detect fetal 
fibronectin (fFN) in a cervico/vaginal sample and/or the result of a test that 
30 determines the level of a local inflammatory product protein in cervico/vaginal 
samples and/or the result of a test that assesses estriol or estretol in saliva; 

the selected variables include the results of the test; and 
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the method measures the risk of delivery in 7 days or few days from 
obtaining the sample for the fFN. 

29. The method of claim 28, wherein the decision support system is a 
neural network. 

5 30. The method of claim 28, wherein the decision-support system has 

been trained using a set of variables that do not include biochemical test data. 

31. In a computer system, a method for assessing the risk for delivery 
in 7 days or fewer days, comprising: 

(a) collecting observation values reflecting presence and absence 
10 of specified clinical data factors and storing the observed clinical data factors in 

storage means of the computer system, the specified clinical data factors 
comprising at least four up to all of the factors selected from the group 
consisting of: 

Ethnic Origin Caucasian, Uterine contractions with or without pain, Parity- 
15 abortions, vaginal bleeding at time of sampling, uterine contractions per hour, 
prior to and No previous pregnancies; 

(b) applying the observation values from the memory means to a 
first decision-support system trained on samples of the specified factors; and 
thereupon 

20 (c) extracting from the first decision-support system an output 

value, wherein the output value is a quantitative objective aid to assess the risk 
of delivery in less than or in 7 days. 

32. The method of claim 31, wherein the decision-support system 
comprises a neural network. 

25 33. The method of claim 31 , wherein at least five factors are selected. 

34. The method of claim 31 , wherein the clinical factors further 
comprise the result of a test that detects fetal fibronectin in cervico/vaginal 
sample and/or the result of a test that determines the level of a local 
inflammatory product protein in cervico/vaginal samples and/or the result of a 

30 test that assesses estriol or estretol in saliva. 

35. The method of claim 34, wherein the selected factors include the 
result of the test. 
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36. The method of claim 35, further comprising: 

b1) applying said observation values from said memory means to a 
plurality of the first decision-support system, wherein each one of the first 
decision-support systems is trained on the samples of the specified factors with 
5 different starting weights for each training; 

c1) extracting from the first decision-support system, output value 
pairs for each one of said first neural networks; and 

d) forming a linear combination of said first ones of said output 
value pairs and forming a linear combination of said second ones of said output 
10 value pairs, to obtain a confidence index pair, said confidence index pair being 
said quantitative objective aid. 

37. The method of claim 36, wherein the first decision support system 
is a neural network that comprises a three-layer network containing an input 
layer, a hidden layer and an output layer, the input layer having seven input 

15 nodes, first, second, third, forth and fifth second hidden layer nodes, a hidden 
layer bias for each hidden layer node, first and second output layer nodes in the 
output layer, and an output layer bias for each output layer node. 

38. The method of claim 36, wherein the first decision support 
system is a neural network and each of the plurality of first trained neural 

20 networks comprises a three-layer network comprising an input layer, a hidden 
layer and an output layer. 

39. The method of claim 31, further comprising: 

b1 ) applying said observation values from said memory means to a 
plurality of the first decision-support system, wherein each one of the first 
25 decision-support systems is trained on the samples of the specified factors with 
different starting weights for each training; 

c1> extracting from the first decision-support system, output value 
pairs for each one of said first neural networks; and 

d) forming a linear combination of said first ones of said output 
30 value pairs and forming a linear combination of said second ones of said output 
value pairs, to obtain a confidence index pair, said confidence index pair being 
said quantitative objective aid. 
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40. The method of claim 31, wherein the first decision support system 
is a neural network that comprises a three-layer network containing an input 
layer, a hidden layer and an output layer, the input layer having six input nodes, 
first, second, third, forth and fifth second hidden layer nodes, a hidden layer bias 

5 for each hidden layer node, first and second output layer nodes in the output 
layer, and an output layer bias for each output layer node. 

41 . The method of claim 31, wherein the first decision support system 
is a neural network and each of the plurality of first trained neural networks 
comprises a three-layer network comprising an input layer, a hidden layer and an 

10 output layer. 

42. In a computer system, a method for assessing the risk for delivery 
in 7 days or fewer days, comprising the steps of: 

(a) collecting observation values reflecting presence and absence 
of specified factors and storing the observation factors in storage means of the 

15 computer system, the specified factors comprising: Ethnic Origin Caucasian, 
Uterine contractions with or without pain, Parity-abortions, vaginal bleeding at 
time of sampling, uterine contractions per hour, prior to and No previous 
pregnancies; 

(b) obtaining results from the patient of a test that detects fetal 
20 fibronectin (fFN) mammalian body tissue and fluid samples, and/or of a test that 

determines the level of a local inflammatory product protein in cervico/vaginal 
samples; and/or of a test that assesses estriol or estretol in saliva; 

(c) applying the observation values and the fFN test results from 
the memory means to a second neural network trained on samples of the 

25 specified factors and the test results; and thereupon 

(d) extracting from the second trained neural network an output 
value pair, the output value pair being a preliminary indicator for the risk of 
delivery in 7 days or few days from obtaining the cervico/vaginal sample. 

43. The method of claim 42, further including the steps of: 

30 (d) applying the observation values and the relevant biochemical 

test results from the memory means to a plurality of the second neural networks, 
each one of the first neural networks being trained on the samples of the 
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specified factors with starting weights for each training being randomly 
initialized; 

(d1) extracting from each one of the first trained neural networks, 
output value pairs for each one of the first neural networks; and 
5 (e) forming a linear combination of the first ones of the output 

value pairs and forming a linear combination of the second ones of the output 
value pairs, to obtain a confidence index pair, the confidence index pair being 
the indicator of the risk for delivery in 7 days or fewer days. 

44. The method of claim 42, wherein the first trained neural network 
10 comprises a three-layer network containing an input layer, a hidden layer and an 
output layer, the input layer having seven input nodes, first, second, third, fourth 
and fifth hidden layer nodes, a hidden layer bias for each hidden layer node, first 
and second output layer nodes in the output layer, and an output layer bias for 
each output layer node. 
15 45. A method for assessing the risk for delivery in 14 or fewer days, 

comprising assessing a subset of variables containing at least three up to all of 
the following variables: 

Ethnic Origin Hispanic; 
Marital Status living with partner; 
20 Uterine contractions with or without pain; 

Cervical dilatation; 
Uterine contractions per hour; and 
No previous pregnancies, 
using a decision-support system that has been trained to assess the risk of 
25 delivery within fourteen days. 

46. The method of claim 45, wherein: 

the variables further include the result of a test for to detect fetal 
fibronectin (fFN) in a cervico/vaginal sample and/or the result of a test that 
determines the level of a local inflammatory product protein in cervico/vaginal 
30 samples and/or the result of a test that assesses estriol or estretol in saliva; 

the selected variables include the results of the test; and 
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the method measures the risk of delivery in 14 days or few days from 
obtaining the sample for the fFN. 

47. The method of claim 46, wherein the decision support system is a 
neural network. 

5 48. The method of claim 46, wherein the decision-support system has 

been trained using a set of variables that do not include biochemical test data. 

49. The method of claim 46, wherein the decision-support system has 
been trained using a set of variables that do not include the results of a test that 
detects fetal fibronectin in cervico/vaginal samples. 
10 50. In a computer system, a method for assessing the risk for delivery 

in 14 days or fewer days, comprising: 

(a) collecting observation values reflecting presence and absence 
of specified clinical data factors and storing the observed clinical data factors in 
storage means of the computer system, the specified clinical data factors 
15 comprising at least four up to all of the factors selected from the group 
consisting of: 

Ethnic Origin Hispanic, Marital Status living with partner, Uterine contractions 
with or without pain, cervical dilatation, Uterine contractions per hour, and No 
previous pregnancies; 
20 (b) applying the observation values from the memory means to a 

first decision-support system trained on samples of the specified factors; and 
thereupon 

(c) extracting from the first decision-support system an output 
value, wherein the output value is a quantitative objective aid to assess the risk 
25 of delivery in less than or in 14 days. 

51. The method of claim 50, wherein the decision-support system 
comprises a neural network. 

52. The method of claim 50, wherein at least five factors are selected. 

53. The method of claim 50, wherein the clinical factors further 
30 comprise the result of a test that detects fetal fibronectin in cervico/vaginal 

samples. 
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54. The method of claim 50, wherein the selected factors include the 
result of the test. 

55. The method of claim 54, further comprising: 

b1) applying said observation values from said memory means to a 
5 plurality of the first decision-support system, wherein each one of the first 

decision-support systems is trained on the samples of the specified factors with 
different starting weights for each training; 

c1) extracting from the first decision-support system, output value 
pairs for each one of said first neural networks; and 
10 d) forming a linear combination of said first ones of said output 

value pairs and forming a linear combination of said second ones of said output 
value pairs, to obtain a confidence index pair, said confidence index pair being 
said quantitative objective aid. 

56. The method of claim 54, wherein the first decision support system 
15 is a neural network that comprises a three-layer network containing an input 

layer, a hidden layer and an output layer, the input layer having seven input 
nodes, first, second, third, forth and fifth second hidden layer nodes, a hidden 
layer bias for each hidden layer node, first and second output layer nodes in the 
output layer, and an output layer bias for each output layer node. 

20 57. The method of claim 54, wherein the first decision support 

system is a neural network and each of the plurality of first trained neural 
networks comprises a three-layer network comprising an input layer, a hidden 
layer and an output layer. 

58. The method of claim 50, further comprising: 

25 b1) applying said observation values from said memory means to a 

plurality of the first decision-support system, wherein each one of the first 
decision-support systems is trained on the samples of the specified factors with 
different starting weights for each training; 

c1) extracting from the first decision-support system, output value 

30 pairs for each one of said first neural networks; and 

d) forming a linear combination of said first ones of said output 
value pairs and forming a linear combination of said second ones of said output 
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value pairs, to obtain a confidence index pair, said confidence index pair being 
said quantitative objective aid. 

59. The method of claim 50, wherein the first decision support system 
is a neural network that comprises a three-layer network containing an input 

5 layer, a hidden layer and an output layer, the input layer having six input nodes, 
first, second, third, forth and fifth second hidden layer nodes, a hidden layer bias 
for each hidden layer node, first and second output layer nodes in the output 
layer, and an output layer bias for each output layer node. 

60. The method of claim 50, wherein the first decision support system 
10 is a neural network and each of the plurality of first trained neural networks 

comprises a three-layer network comprising an input layer, a hidden layer and an 
output layer. 

61 . In a computer system, a method for assessing the risk for delivery 
in 14 days or fewer days, comprising the steps of: 

15 (a) collecting observation values reflecting presence and absence 

of specified factors and storing the observation factors in storage means of the 
computer system, the specified factors comprising: Ethnic Origin Hispanic, 
Marital Status living with partner, Uterine contractions with or without pain, 
cervical dilatation, Uterine contractions per hour, and No previous pregnancies; 

20 (b) obtaining result from the patient of a test that detects fetal 

fibronectin (fFN) mammalian body tissue and fluid samples and/or the result of a 
test that determines the level of a local inflammatory product protein in 
cervico/vaginal samples and/or the result of a test that assesses estriol or 
estretoi in saliva; 

25 (c) applying the observation values and the fFN test results from 

the memory means to a second neural network trained on samples of the 
specified factors and the test results; and thereupon 

(d) extracting from the second trained neural network an output 
value pair, the output value pair being a preliminary indicator for the risk of 

30 delivery in 14 days or few days from obtaining the cervico/vaginal sample. 
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62. The method of claim 63, further including the steps of: 

(c1) applying the observation values and the relevant biochemical 
test results from the memory means to a plurality of the second neural networks, 
each one of the first neural networks being trained on the samples of the 
5 specified factors with starting weights for each training being randomly 
initialized; 

(d1) extracting from each one of the first trained neural networks, 
output value pairs for each one of the first neural networks; and 

(e) forming a linear combination of the first ones of the output 
10 value pairs and forming a linear combination of the second ones of the output 
value pairs, to obtain a confidence index pair, the confidence index pair being 
the indicator of the risk for delivery in 14 days or fewer days. 

63. The method of claim 64, wherein the first trained neural network 
comprises a three-layer network containing an input layer, a hidden layer and an 

15 output layer, the input layer having seven input nodes, first, second, third, fourth 
and fifth hidden layer nodes, a hidden layer bias for each hidden layer node, first 
and second output layer nodes in the output layer, and an output layer bias for 
each output layer node. 

64. The method of claim 10 wherein the decision-support system has 
20 been trained using a set of variables that do not include the results of a test that 

detects fetal fibronectin in samples of mammalian body tissue and fluids. 

65. The method of claim 10, wherein the set of variables further 
includes the result of a test that detects fetal fibronectin in cervico/vaginal 
samples. 

25 66. The method of claim 13, wherein the clinical factors further 

comprise the result of a test that detects fetal fibronectin in mammalian body 
tissue and fluid samples. 

67. The method of claim 24, wherein the sample is a cervico/vaginal 
sample. 
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68. The method of claim 27, wherein: 

the variables further include the result of a test for to detect fetal 
fibronectin (fFN) in mammalian body tissue and fluid samples and/or the result of 
a test that determines the level of a local inflammatory product protein in 
5 cervico/vaginal samples and/or the result of a test that assesses estriol or 
estretol in saliva; 

the selected variables include the results of the test; and 
the method measures the risk of delivery in 7 days or few days from 
obtaining the sample for the fFN. 
10 69. The method of claim 31, wherein the clinical factors further 

comprise the result of a test that detects fetal fibronectin in mammalian body 
tissue and fluid samples. 

70. The method of claim 42, wherein the clinical factors further 
comprise the result of a test that detects fetal fibronectin in mammalian body 

15 tissue and fluid samples and/or the result of a test that determines the level of a 
local inflammatory product protein in cervico/vaginal samples and/or the result of 
a test that assesses estriol or estretol in saliva. 

71. The method of claim 45, wherein: 

comprise the result of a test that detects fetal fibronectin in mammalian body 
20 tissue and fluid samples and/or the result of a test that determines the level of a 

local inflammatory product protein in cervico/vaginal samples and/or the result of 

a test that assesses estriol or estretol in saliva the selected variables include 

the results of the test; and 

the method measures the risk of delivery in 1 4 days or few days from 
25 obtaining the sample for the fFN. 

72. The method of claim 50, wherein the clinical factors further 
comprise the result of a test that detects fetal fibronectin in mammalian body 
tissue and fluid samples and/or the result of a test that determines the level of a 
local inflammatory product protein in cervico/vaginal samples and/or the result of 

30 a test that assesses estriol or estretol in saliva. 

73. The method of claim 61, wherein the sample is a cervico/vaginal 
sample and the test detects fetal fibronectin. 
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